· occlusion resolution operators for threedimensional detail-in-context david j. cowperthwaite...

OCCLUSION RESOLUTION OPERATORS

FOR THREEDIMENSIONAL

DETAIL-IN-CONTEXT

David J. Cowperthwaite

B.Sc., York University, 1994

A THESIS SUBMITTED IN PARTIAL FUCFILLMENT

OF THE REQUIREMENTS FOR THE DEGREE OF

DOCTOR OF PHILOSOPHY in the School

of

Camputing Science

@ David J. Cowperthwaite 2000

SIMON FR4SER U m R S I T Y August 2000

AU rights reserved. This work may not be

reproduced in whole or in part, by pbotocopy

or ot her means, without the permission of the aut hor.

Acquisitions and Acquisitions et Bibliographe Setviis s e W s bibliographiques

The author has granted a non- exclusive licence aüowing the National Liôrary of Canada to reproduce, loan, distribute or sel copies of this thesis in microfom, paper or electronic formats.

The author retains ownership of the copyright in this thesis. Neither the thesis nor substantial extracts fiom it may be printed or otherwise reproduced without the author's permission.

L'auteur a accordé une licence non exclusive permettant il la Bibliothèque nationale du Canada de reproduire, prêter, distribuer ou vendre des copies de cette thèse sous la forme de microfiche/^ de reproduction sur papier ou sur fonnat électronique.

L'auteur conserve la propriété du droit d'auteur qui protège cette thèse. Ni la thèse ni des extraits substantiels de celle-ci ne doivent être imprimds ou autrement reproduits sans son autorisation.

Abstract

The inionnation explosion is cbanging the daily lives of thc "wird ' population.

Increasing numbers of individuals are being empowered to act as their own &or-

mation broken, interacting with large and expanding data spaces, for example the

World Wide Web. Scientists tao are dealing with growing databases of empincal and

simulateci information. New tools for visualizing these spaces are being dewloped

which incorporatc threc-dimensional visual repmentations of information. These

three-dimensianal representations are believed to leverage the individual's capacity

for comprehension and navigation in our threedimeosional world. In practice one is

faced with the inherent limitations of 2D presentation and interaction through the

traditional twedimensional desktop computer display.

The spatial limitations of the two-dimenuional display (referred to as the "screen

real-estate problem" ) have motivated the development of detail-in-context methuds of

information presentation and exploration. Much of the work in this field has concen-

trated on presentation methods for 2D information spaces. While a few techniques

have incorporated 3D interaction metaphors, such as surfaces which produce mag-

nifia tion through perspective distort ion, fewer still bave focused on techniqua for

interaction with 3D representations of information. Three-dimensional representa-

tions of information present specific challenges not found in 2D representations, for

example the effect of occlusion on the visibility of elements. 'lladitiond approaches to

dealiag with occlusion in three-dimensional representations include techniques such

as cutting planes, viewer navigation, filtering of information and transparency. Wnile

these methods provide clearer visual access to elements of interest it is often at the

expense of removing rnuch of the contextual information h m a representation.

We present a technique which employs a new approach to some of the challenges in

interacting with 3D reptesentations of information. Speciiidy we resolve occlusion

of objects in a 3D scene through a layout aàjustment algorithm derived h m 2D detail-in-context viewing methods. Our extension beyond traditional 2D approaches

to layout adjustment in 3D accounts for the specific challenges of occlusion in 3D

representations, where other such extensions do not. In doing so we provide a simple

yet powerful tool for providing non-occluded views of objects or regions of interest

in a 3D information representation with minimal adjustment of the original stmcture

and without the use of cutting planes, transparency or information filtering. We cal1

these operators Occlusion Reùucing ïlansformations (Onfs).

For Julie, my wife.

"A Iit tle learning is a dangerous thiog;

Drink deep, or taste not the Pierian spring:

There sbdow dmugh ts in taxica te the brain,

And dinking largely sobers us again."

- An Essay on Criticism ALEXANDER POPE, 171 1

Contents

Approval ii

Abstract iii

Dedication v

Quotat ion v i

List of Tables x

List of Figures xii

1 Introduction 1 1.1 Visualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.2 Three-Dimensional Information Graphics . . . . . . . . . . . . . . . . 3 1.3 Cost of 3D Representations . . . . . . . . . . . . . . . . . . . . . . . 5

1.4 Detail-in-Context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

1.5 ThesisStatement . . . . . . . . . . . . . . . . . . . . . . . . . . . . - 7 1.6 Thesis Outline . . . . . . . . . . . . . . . . . - - - . . . . . . . . . . . 8

2 Related Work O

2.1 3D Perceptual Cues . . . . . . . . . . . . . . . - . . . . . . . . . . . . 10

2.2 Three-Dimensional Visualization . . . . . . . . . . . . . . . . . . . . . 12

2.2.1 Scientific Visuaiization . . . . . . . . . . . . . . . . . . . . . . 13

2.2.2 Information Visualization . . . . . . . . . . . . . . . . . . . . 17

. . . . . . . . . . . . . 2.3 Structural Framework for Visualization Design 21 . . . . . . . . . . . . . . . . . . . . 2.4 Methods for Occlusion Reduction 22

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.1 Navigation 22 . . . . . . . . . . . . . . . . . . . . . . . 2.4.2 Partial Tkansparency 24

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.3 Culling 25 . . . . . . . . . . . . . . . . . . . . . . . . . 2.5 3D Deformation Methods 27

. . . . . . . . . . . . . . . . . . 2.5.1 Space Deformation Operators 28 . . . . . . . . . . . . . . . . . . . . . . . . . 2.5.2 Zoom Illusfirator 29

. . . . . . . . . . . . . . . . . . . . . . . . . . 2.5.3 Page Avoidance 30 . . . . . . . . . . . . . . . . . . . . . . . . 2.6 Detail-in-Context Viewing 31

. . . . . . . . . . . 2.6.1 Perspective-Based Fisheyes for 2D layouts 31

3 Method 44 . . . . . . . . . . . . . . . . . . . . . . . . 3.1 Layout Adjustrnent in 2D 45 . . . . . . . . . . . . . . . . . . . . . . . 3.2 Occlusion and the Sight-line 45

. . . . . . . . . . . . . . . . . . . . . . . . 3.2.1 Towards a Solution 47 . . . . . . . . . . . . . . . . . . . . . . . 3.2.2 Redefining the Focus 49

. . . . . . . . . . . . . . . 3.2.3 ORT-Relative Coordinate Systems 53 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Distortion Space 55

4 Applications 82 . . . . . . . . . . . . . . . . . . . . . . 4.1 Discrete Data Representations 63

. . . . . . . . . . . . . . . . . . 4.1.1 Regilar 3D Graphs Structures 64 . . . . . . . . . . . . . 4.1.2 General3D Node and Edge Structures 67

. . . . . . . . . . . . . . . . 4.1.3 Hierarchical3D Graph Structures Ti? . . . . . . . . . . . . . . . . . . . . 4.1.4 3D Desktop Environment 74

. . . . . . . . . . . . . . . . . . . . 4.2 Contiguous Data Representatioris 78 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.1 3D Models 80

. . . . . . . . . . . . . . . . . . . . . . . . 4.2.2 Isosurface Set Data 86 . . . . . . . . . . . . . . . . . . . . 4.3 Continuous Data Representations 89

. . . . . . . . . . . . . . . . . . . . . . . 4.3.1 Fast-SplatRendering 93

viii

4.3.2 3D Texture-Based Rendering . . . . . . . . . . . . . . . . . . 97

4.3.3 Temporally Sequential 2D information . . . . . . . . . . . . . 113

4.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116

5 Conclusion 118

5.1 Contribution . . . . . . . . . . + . . . . . . . . . . . . . . . . . . . . . 119

5.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

5.3 Final Thought . . . . . . . . . . . . . . . . . - . . . . . . . . . . . . . 120

A 3D Perception 121 -4.1 Perccptual Cucs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

B Marching Cubes 129

Bibiiography 133

List of Tables

2.1 A visual cornparison of a range of magnification functions (Constant -a

j ( x ) = 1, Linear f(x) = 1 - x, Gaussion f(x) = e y , HemiÉ;phere

f (x) = sin(ccish(1 - x)) , Cosine f (x) = w ( x * ), Tangent f ( x ) = ian(O.l*z*f)

L - yn(o.!xi*j) and Inverse Hemisphere f(x) = 1 - sin(cosh(1 - x))) and their pmperties of slope 5, apparent planar translation t ( x ) and

rcsulting magnification m(x) (within a penpective-distortion systcm

such as 3DPS). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

Illustration of the application of four conmon 2D detail-in-context lay-

out adjustment approaches to 3D layout via simple inclusion of 3Td di-

mension. In the Rrs t row are examples of step and non-linear orthogonal

stretching, non-linear radial displacement and non-space-filling orthog-

onal stretching. Row two illustrates the effect of moving from (2, y) to

(x, y, z ) for data and displacement function. The third row shows the

effect of the layout function without the accompanying magnification

of nodes. Row four shows the displacement ody efiect extended into

three dimensions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

The &ect of adding the e k t of an ORT to the 3D extensions ofsome

common 2D layout adjustment scbemes for detail-in-context viewing.

In the first row of images we see the simple extension of the approaches

to 3D, the central focal point is even more occluded than before the

layout adjustment in most casa. The second row adds the operation

of an ORT to clear the line of sight h m the viewpoint to the focal point. 53

3.3 The SuperQuadric distance metric allows separate specification of the

ns and ew shaping parameten to achieve a wider range of possible

metric spaces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

3.4 Parameters available in the definition of O M operators. . . . . . . . 59

3.5 Some of the space of ORT specifications possible by varying the source

and distribution of the operator. The left column illustrates OWs defined relative to the z-axis of the O W CS, the right column illustrates

OWs defined relative to the y = O and x = O plane of the 0W CS. . ûû

List of Figures

Image-ordcr volume rendering proceeds across scan-lines, tracing the

path of rays through the volume and performing shading calculations

at regular intervals or at intersections with the data grid. . . . . . . . Data and normal values rnust be interpolated from cell vertices to points

within a ceii. Three linear interpolations are used to accomplish this;

dong edges, then across faces, then through the cell. . . . . . . . . . Object-order volume rendering methods traverse the data set and deter-

mine the contribution of each element to the final image. Object-order

methods may operate with front to back (Under operator) or back to

front (Over operator) composition. . . . . . . . . . . . . . . . . . . . Structural classes of threedirnensional representations . . . . . . . . . Moving the viewpoint makes the highlighted bone (the fint metatarsal)

occluded in (a) visible in (b). . . . . . . . . . . . . . . . . . . . . . . Mavernent of the viewpoint into the structure puts elements that oc-

cludcd the nodc of interest in (a) behind the viewer in (b). . . . . . . Partial ~ a n s p u e n c y . . . . . . . . . . . . . . . . . . . . . . . . . . . Cutting planes and regions remove volumetric data in a half-space (a)

or subvoliime (b) from the final image and make previously occluded

. . . . . . . . . . . . . . . . . . . surfaces visible adjacent to the cut.

Selective removal of component group im provcs visibilit y of remaining

components. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2.10 The effect of a warp operator on the path of a ray through a scene. The

deficcted ray results in the appearance of a deformation of the surface.

(After [61]) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.11 Discontinuous ray deflectors operate by deflecting rays in opposite di-

rections from opposite sides of a plane. Ray sampling is restricted to

the original side of the plane, thereby producing a cutting and retract-

. . . . . . . . . . . . . . . . . . . . ing form of distortion. (After [El])

2.12 The base configuration of the Perspective Wall with no regions of in-

terest (a). The surface has one dominant Iinear dimension and is at a

. . . . . . constant depth in z in the perspective viewing frustum (b).

2.13 Perspective Wd with a single ROI specified in the middle of the field of

view, generating a region of increased scale and surrounding distorted

regions (a). The ROI is at a constant depth in z with respect to the

viewpoint in the perspective viewing frustum (b). . . . . . . . . . . . 2.14 The features of the perspective viewing hstum. The h s t u m foms

a pyramid with the viewpoint at the apex. The far-pianc forms the

bue of the pyramid. The width and height of the pyramid are usually

defined by the field of view and the aspect ratio. The field of view

is the angular horizontal width of the pyramid, and the aspect ratio

defines the relationship between the width and the height. (a = e). Objects within the pyramid are visible in perspective projection if they

are located between the near and fm planes in depth. The central axis

of the pyramid dehes the direction of the view in world coordinates.

2.15 Geometry of the frustum is sheared in x to keep the Viewpoint directly

over the offcenter ROI. . . . . . . . . . . . . . . . . . . . . . . . . . . 2.16 Effect of simple vertical movement of a portion of the surface in an

off-center lens after perspective projection (a). The r e m n is that the

surface now extends outside of the perspective viewing frustum (b). . 2.17 Shearing the distorted region sa that it is orienteci towards the view-

point (a) brings the entùe extent of the lens back into the projected

image(b). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

xiii

2.18 Shearing the lenses rather than the viewing frusturn aiiows for the

specification of multiple ROI (a). The area of intersection of lenses must

be blended to provide a smooth transition between the two shearing

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . directions (b).

2.19 The Gaussian curve f (x) = e-'O-O1 used in Three-Dimensional Pli-

able Surfaces to provide smooth integration of the ROIS and original

. . . . . . . . . . . . . . . . . . . . . . . . . . . . information layout.

2.20 After perspective projection, the apparent transformation t(x) of points

on a surface transformed by the application of a gaussian lens with a

maximum height of 1 and a viewpoint distance of 2 from the original

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . surface plane. 10.02'

2.21 The displaced position of points d(x) = x + 2-c-10.021. as a rcsult of the

gaussian lens after perspective projection. . . . . . . . . . . . . . . . 2.22 The magnification (with single-point perspective projection) and com-

pression distribution as a result of the gaussian lens. m(x) = % . . 2.23 The progression of Lp distance metrics from L1 (figure 2.23(a)) to L200

(figure 2.23(d)) in two dimensions. . . . . . . . . . . . . . . . . . . .

3.1 Operation of lineu ORT in crwsection. Focal point and viewpoint

define the line of sight through the structure (a). Distance of other

elements to line of sight determines direction of displacement (b). The

length of vectors in (b) will form the input into the hnctioa which

determines the magnitude of the resulting displaccmcnt vecton. Final

transformed layout produces clear line of sight from viewpoint to focal

point (c). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Increasing degree of application of two ORTs to reveal two objects of

interest (highlighted here with darker color) in a 3D graph layout. . . 3.3 Rotation of the 3D graph to illustrate the occlusion of two objects of

interest (nodes highlighted with darker color). A clear view of even the

nearer of the two in the structure is available h m only a limiteci range

of viewpoints. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

xiv

The same three viewpoints and same two objects of interest (now high-

lighted and increased in scale for emphasis) with the application of

ORTs. Even the node at the far side of the graph is visible through

. . . . . . . . . . . . . . . . the sight-line-clearing effect of the ORT. 52

Annotateci framework of diagrams illustrating the relative shape of a

selection of ORT functions. On the left (a) the O W Coordinate System

(CS) z-axis is aligncd with the World CS z-ais, on the right (b) the

camera position (VRP) bas been moved and the ORT CS is re-oriented

. . . . . . . . . . . . . . . . . . . . . . . . . . . to track the change. 54

Schematic of orthogonal stretch OW. Distance of points is rneasured

to nearest of the 3 planes passing through the focal point. . . . . . . 56

Linear extrusion through z axis of the functions describing the oper-

ation of a detail-in-context layout adjustment scherne. The Gaussian

. . . . . . . . . . . . . . . . . . cuwe f (z) = e-10-0z2 forms the basis. 56

The sarne graphs now illustrating the effect of linearly scaling the a p

plication of the basis function according to depth in 2. . . . . . . . . 57

A secondary shaping function applied to the horizontal plane-relative

ORT. Scaling in z is constant but the addition of the shaping curve

can be used to constrain the extent of the plane-relative function in x. 57

3.10 Distance measurement according to the Lp metric in the-dimensions. 58

4.1 Three classes of three-dimensional data representations . . . . . . . . 62

4.2 The original layout of the 9 x 9 x 9 3D grid-graph . . . . . . . . . . . 64

4.3 The orthogonal stretch algorithm aligned to the principle planes of the

data layout space (a) and aligncd to the viewer as an ORT operator (b). 65

4.4 The 3D grid-graph with the central node specified as the object of

interest. An 0RT has been applied to reduce occlusion. Color of the

remaining nodes in the graph represent the degree to which they have

been displaceci by the ORT. The darkest nodes have been moved the

most . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

Examples of constant and linear scaling of the application of the ORT dong the z axis of the ORT coordinate system. The constant scaiiig

isolates the object of interest against an empty background while the

linear scaling looks very similar to the line segment relative application. 67

OElT functioas applied relative to a horizontal plane through the o b

ject of interest. Objects within the plane remaia in plane whiie those

above and below arc displaced. la (a) the operator is data-axis relative,

and does not track changes in the viewpoint. The operator in (b) is

viewpoint aligneci. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

Cafieine Molecule: CeHioN40z . . . . . . . . . . . . . . . . . . . . . . 69

Movement of the viewpoint around the calFeine rnolecule without the

application of any ORT functions. . . . . . . . . . . . . . . . . . . . . 69

The q g e n atom indicated in (a) is selected as the atom of interest for a

linear-source ORT. The same movement of the viewpoint is performed

around the caffeine molecule and this atorn remains visible as other

atoms are deflected away h m sight-Lie. . . . . . . . . . . . . . . . 70

4.10 Sequence illustrating the application of a Fiear-source ORT to the

structure of vitamin B12. The Oxygen atom selected as an atom of

interest is in the region indicated by the overlay box. . . . . . . . . . 71

4-11 .4 detail view of the region indicated by the overlay box in the previous

figure. The result of the successive application of a linear-source ORT to the (initially hidden) Oxygen atom is illustrateci, . . . . . . . . . . 71

4.12 .4 selected leaf-node in a cone tree layout of a directory structure is

indicated by the overlay in (a). This node is brought to the front

through concentric rotations of the cone tree structure; (b) through (d) 72

4.13 Two leaf-nodes, labelleci a and b in (a) are selected simultaneously.

Application of two ORT operators improves the visibility of these nodes

without explicitly rotating one or the other to the front; (b) and (c). . 73

4.14 Once ORT operators are attached to nodes a and b, in (a), these nodes

remain visible during movement of the viewpoint; (b) and (c). . . . . 74

4.15 The area of influence and viewpoint alignment of the O W operators

in the previous sequence, as seen from a secondary viewpoint. The

OFfï operaton remah digned to the primary viewpoint as it is moved

around the cone tree. . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.16 3D Desktop environment . . . . . . . . . . . . . . . . . . . . . . . . . 4.17 As the seiected window is pushed back through a cluster of windows in

the 3D desktop environment the cluster is disperseci in order to prevent

occlusion of the selected window. . . . . . . . . . . . . . . . . . . . . 4.18 Amotated images from the previous sequence illustrating the initial

position (boxes) and movement (arrows) of the seiected (solid iine)

and other (broken line) windows. . . . . . . . . . . . . . . . . . . . . 4.19 As the selected window is moved from its initial pmition in the upper

left of the view the cluster of other windows which it passes in front of

are d i s p e d by the action of the ORT attached to the setected window.

4.20 Annotation of two frames fiom the previous sequence. As the selected

window moves from position a to position b the remaining windows arc

deflected by the action of the ORï. The mow clusters in (a) indicate

the progression of deflection vectors for the remaining windows. Early

to late vectors in the resulting motion are shaded h m dark to lighter

grey. On the right (b) illustrates the state of the layout at the midpoint

of the sequence. Initial (grey boxes) m d final (black boxes) positions

of the windows are indicateà as well as their resulting displacements

(arrows) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.21 The skeletal model of the foot used in the following example. This

mode1 contains 26 separate components and 4204 triangular faces. . . 4.22 The external cuneiform bone (circled in (a) and highiighted in sll im-

ages) is selecteà as the focus and an OElT operator is used to displace

the remaining 25 bones away from the sight-line. . . . . . . . . . . . 4.23 Again the extemal cuneiform is the object of interest and remains vis-

ible in this sequence as the vicwpoint moves around the model. . . . .

4.24 Again the extemal cuneiform is the object of interest and remains v i s

. . . . ible in this seguence as the viewpoint moves around the model. 83

4.25 In (a) no scaling is applied, the effect of the O W is simply to displace

components and d u c e occlusion. In (b) we have subsequently scaled

components according to their geometric distance €rom the object of

interest, the extenid cuneiform bone. . . . . . . . . . . . . . . . . . 84

4.26 Figure (a) illustrates the basic configuration of the perspective viewing

volume and 3D model. Spheres indicate the location of the viewpoint,

the view reference point and the point midway between. Components

of the model are translated dong their individual lines of sight in (b)

. . . . . . . . . . to produce magniôcation via perspective projection. 85

4.27 The effect of decreasing the distance d t'rom the viewpoint on projecterl

scale in perspective projection. Final scale varies as the inverse of the

change in distance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

4.28 Side (a) and front (b) views of the foot model with the navicular bone

selected as an objcct of interest and higblighted. No distortion or mag-

nification hw been applied and the bone remllins dl but mmpletely

occluded in these two views. . . . . . . . . . . . . . . . . . . . . . . . 86

4.29 Side (a) and front (b) views of the foot model with the navicular bone

selected as an object of interest and highlighted. Distortion only bas

been applied to the layout of the model, with no scaling for emphasis. 87

4.30 The navicular bone is selected as an object of interest and an OEtT is

applied to reduce occlusion. Simultaneously a small degree of magnifi-

cation has been applied to emphasize the navicular bone and its neigh-

borhood. Mapification here is produced through perspective transfor-

mation and as a result the navicular is rendered in Gont of other bones

that may have stiil resulted in partial occlusion. . . . . . . . . . . . . 88

4.31 The same two view of the human foot with the navicuiar bone as an

object of interest in the layout. Here magnification is produced via in-

place scahg of the individual components. The most apparent different

is that in (b) the interior cuneiform bone now partially occludes the

navicular. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.32 A detail view of the area just in front of the navicular bone with in-

place scding [a) and perspective scaling (b). The intersection of the

external cuneiform and the third metatarsat in (a) is resolved in (b) by

the relative displacement of the components in depth. . . . . . . . . . 4.33 Example source images for the generation of Marching Cubes derived

surfaces of MRi data. . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.34 Three separate surfaces from diagnostic MRI data of MultipleSclerosis

(MS) lesions. Proton-Density layeca (a) reveal outer surfaces such as

the skin, T2 layers (b) reveal neural tissue (brain and eyes), while the

lesion mask (c) indicates location of MS lesions. These three data

sets are used in the dcmonstration of the application of an ORT to

volumetric data visualization. . . . . . . . . . . . . . . . . . . . . . . 4.35 Composite 4.35 is rendered as slightly transparent in order to make

spatial organization apparent. . . . . . . . . . . . . . . . . . . . . . . 4.36 Sequence illustrating the application of an ORT to imurface data. The

lesion mask layer (green) is not affected by the scaled and truncated

planar deformation and is revealed as the outer layen are cut and

pushed back. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.37 UNC head data set rendered via fast-splatting . . . . . . . . . . . . . 4.38 The application of a vertical-plane-source O W to CT data of a human

skuil rendered via fast splatting. Ohserve the increase in brightness at

the edge of the ORT-induced split. This is the result of splat primitives

overlapping. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

4.39 A horizontal-plane ORT applied to the UNC Head data set. In (a) the

OKï' is aligned to the viewpoint. In (b) we have moved the viewpoint

indepeudent of the OItT (disabled automatic tracking of the viewpoint)

in order to illustrate the linear scaling of the application of the ORT in

view-aligneci depth. The ORT is scaled in depth from the front of the

representation to the depth of the region of interest. . . . . . . . . . . 4.40 The same data set and orientation of views. Here a shaping curve

has b e n added to control the extent of the O W operator across the

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . horizontal plane.

4.41 The Visible Human Female data set with a plane-relative ORT applied.

Here the ORT scaled in depth from the front to back of the data set,

. . . . . . . . . . rather than from the front to the region of interest.

. . . . . . . . . . . . 4.42 Relation of slice domain to volume data domain.

4.43 Two basic approaches to the alignment of slices in 3D-Texture hardware

accelerated volume rendering. . . . . . . . . . . . . . . . . . . . . . . 4.44 2D Gaussian Function j(s) = e-10.0'2-10.0* . . . . . . . . . . . . . . 4.45 Hessian of 2 D Gaussian Function f (r) = e-10.022-10-0y2 . . . . . . . . 4.46 Anisotropic mesh aligned to Hessian of Gaussian function. . . . . . . 4.47 Sampling planes aliped to data space axis (a) or centered an sight-line

(b) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.48 Configuration of tessellateci plane and hidden tcxturc surface used in

demonstrating stretch approach to OnT application. . . . . . . . . . 4.49 Progressive application of deformation and resulting transparency ef-

fect. As triangles are stretched they are made progressively less opaque.

The resuit is that in the area of the deformation the background layer

becomes visible. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.50 Detail view illustrating the transition of opacity values at the boundary

of the deformation which r e d t s in the blurry appearance. . . . . . .

4.51 The initial configuration of the slice sampling mesh. Triangulation

density is increased in the inside corner where OEM' displacements will occur. This minimizes the extent of linear interpolation of texture

coordinates. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.52 introduction of a semi-circular deformation of the texture sampling

. . . . . . . . . . . . . . mesh by deforming vertices dong the y axis.

4.53 Mirroring the single deformed texture sample plane ailows the creation

. . . . . . . . . . of a closed ernpty region in the middle of the plane.

4.54 OpenGL clipping planes are used to trim the texture planes to the

. . . . . . . . . . . . . . boundaries of the volume presentation space

4.55 Progressive application of ORT to produce a horizontal, shaped, open-

. . . . . . . . . . ing in a single plane in a volumetric representation.

4.56 Progressive application of O W to produce a vertical, shaped, opening

. . . . . . . . . . . . in a single plane in a volumetric representation.

4.57 Increasing the width of the shaping function to enlarge the horizontal

. . . . . . . . . . . . . ORT in a single slice of a volumctric data set.

4.58 Texture transfomation rnatrix is manipulated so that as the intersec-

tion of the sampling planes is moved across the presentation space the

. . . . . . . . . . . . . . . . . . . . texture space remains stationary.

4.59 The Visible Human Maie data set rendered via 3D-texture slicing. . . 4.60 The application of a horizontai OKl' to the Visible Human Maie data

set. The point of interest is behind the left eye and the effect of the

ORT is to reveal two cut-surfaces aligned to the viewpoint without the

removal of data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.61 ,4 more centrally located point OF interest is specified in the Visible

Human Male data set and the viewpoint is moved around the head

. . . . . . . . . . . . . . . . . . . . . from the front to the left side.

4.62 The UNC Head CT data set with vertically and horizontaily aligned

0Kï functions applied to reveal cut surfaces aligned to the current

viewpoint . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

111

I l l

4.63 Arrangement of spatio-temporal data as a Sdimensional cube by using

a spatial axis to rcpresent timc . . . . . . . . . . . . . . . . . . . . . . 4.64 A block of spatio-temporal landscape data and an ORT operator a p

plied to reveal the state of the landscape at an instant in time . . . . . 4.65 Positionhg a split in a data-cube (left). applying an O W operator to

reveal two intemal faces (middle left). repositioning the viewpoint to

obtain a more perpendicular view of the right face (middle right) and

. . . . . . finally selecting a new point in at which to position the split

. . . 4.66 Operation of the book mode ORT with the hardcover appearance

. . . . . . . . . . . . . . A.l Wireframe images with no depth information

A.2 Image containing several depth cues . . . . . . . . . . . . . . . . . . . A.3 Perspective Illusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.4 Stereo Viewing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.5 The simulateci view from the left and right eye. including depth of field

and perspective foreshortening effects . . . . . . . . . . . . . . . . . . . A.6 Texture gradient effect . . . . . . . . . . . . . . . . . . . . . . . . . . A.? StemPair . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.8 Floating region . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

B.l .4 simple equipotential surface through an implicit model . . . . . . . . . . . . . . . . . . . . B.2 UNC Head CT data set rendered as an isosurface

B.3 The United States National Library of Medicine Visible Human Project

data sets . The male B.3(a) and female B.3(b) data sets are derived

from axial slices of the visible data . This data obtained from tbe

NPAC/OLDA online data source . . . . . . . . . . . . . . . . . . . . . 6.4 The 15 basic cases of edge c d n g s in the Marching Cubes algorithm .

Chapter 1

Introduction

By now, the "information explosionn is not new to anyone involved in information

sciences. It is a part of the everyday experience of those for whom the internet has

become integral to their daily lives. News, communications, consumer information

and entertainment, business and financial transactions al1 form the information space

made available through the interface of our personal cornputers (PCs) via the mrld

wide web (WWW) (71. The enormous number of sources and forms of data has

put cr serious strain on the capacity of individuais for finding and utilizing relevant

information.

hotber result of the direct connection of PCs to the WWW is that it has ernpow-

ered users to act as their own information brokers. Traditionally when investigating

a topic we would approach a domain cxpert, describe out situation and allow them to

asyist us in obtaining the relevant information. This has been true in market resemh,

financial analysis and planning, as weii as in consumer-oriented twks such as product

research and travel planning. With the UiWW we are told that this information is al1

a "click away" h m our desktop PC. What the user experiences is a torrent of infor-

mation available on a su bject delivered to them througb increasingly high-bandwidth

connections to the WWW. What users are missing, howewr, is filtering, analysis and

presentation provided by the domain expert information broker.

Information ovetload is not merely a problem for those using the WWW. Busi-

nesses and scientific institutions are amashg huge databases of information for study

and analysis. The rate of accumulation of su& data and the increasingly cornplex

and abstract information space it represents &ives the demand for the dcvelopment

of new tmls for storing, processing and presenting this data in order to make it useful

and meaningful.

Visuaiization in cornputhg is the proces9 of applying computer graphics techniques

to the problem of presenting information to usen. ORen visualization is divided into

two btoad fields: scientific visualization and information visualization. Eacb of these

fields may be further subdivided into more specific domains; for example scientific

visualization encompasses volume visualization, flow visualization, medical visualization etc. Scientific visualization generally involves data which pcaews a mapping to

some real physical space. That is to say that it often deais with information which

is the product of mearement or simulation of a rd-world process. This presents a

natural mapping of the resuiting data to a visual, structural representation, that is

closely related to the source of the data. On the other band information visualization

may deal with much more abstract sources which have no such natural mapping and

therefore require novel repmntations for the data.

Both fields of visualizatioo are more generaily concerned with the creation of vi-

sua1 representations of information that will support understanding and analysis, and

promote insight. This often means developing a visual representation or metaphor for

a non-visuai phenornenon. A graph is an excellent example of mapping the semantic

connections of an information structure (a cornmon example is links between pages

on the WWW) to a visual structure. Such a structure is a visual formdisrn [45]. A

formalism must be leamed to be understood, and it cannot be asnirned to be universal

as its meaning may differ by gmup or culture. For example, the symbolic meanings associated with specific colors vary between cultures. In western cultures white is

a color which evokes images of purity, whereas in eastern cultures it is most often

associated with mourning and loss. [ l l O I . Other examples of visual representations

CHilPTEX 1. INTRODUCTION 3

of information include abstractions of numerical data in large tables and geogtaphi-

cal representations (encoding information as position, color or height on a surface).

There are of course many other examples of visualization techniques, an increasing

number are presented each year, but they al1 have in common the transformation of

information into a visual representation.

Visualization as a whole is not a new activity, neither is it a by-product of the

information age and the development of the computer. Humans have been visualizing

even abstract information for thousands of years. Ancient Egyptian tomb paintings

of the universe, with the star-covered body of the sky-goddess Nut arching over the

earth separating the world €rom the chaos beyond, are visual representations of that

culture's abstract mode1 of the cosmos. The medical and technical illustrations of

Leonardo DaVinci are exquisite examples of early scientific visualization. Leonhard

Euler, a Swiss mathematician, developed the visual formalisms of Euler circles, which

later evolved into Venn diagram and graphs.

1.2 T hree-Dimensional Informat ion G raphics

The evolution of information graphics from traditional print media, through film

and finatly, to the computer display has introduced a new dimension of interactivity

to visualization tools. Animation and cinematography made dynamic information

presentations possible before the advent of computer-generated graphics but they

were generaily non-interactive, play-only, as well as time-consuming to produce and

modify.

Computer graphics is making use of interactive three-dimensional representations

of data in visuaiization increasingly common. Although traditional media are still

used to produce more concrete thredmensional representations as well as 3D pre-

sentations through the construction of physical models, these can be more costly to

produce and difficult to edit or reuse.

Wi t iona l twodimensionai media may aiso present 3D data as 2D projections.

CHAPTER 1. INTRODUCTION

1.3 Cost of 3D Representations

There are a number of costs associated with the use of three-dimensional representa-

tions in visualization: first the additional computational complexity of storing a third

dimension of data and rendering 3D scenes onto a 2D display, second the additional

attention to perceptual cues associated with the display of the 3D scene when viewed

on a 2D display, and third the possibility that some elements will be occluded by

others.

In answer to the first point, the availabilityof increasingly powerful and economical

grapbics accelerators addresses a large part of the computational cost of producing

and interacting with 3D visual representations. At the same time software which

facilitates the creation of 3D visual representation is being developed to leverage the

capabilities of the modern PC. With respect to the second hue, we are able to draw on the understanding of

the operation of human perceptual mechanisms from other fields such as cognitive

psychology. in doing so we cm tune our use of 3D reprcsentations to provide greater

expressive power. Works such as those of Tufte [103] illustrate the potential use, and

misuse, of these mechanisms.

As to the third cost, occlusion is one phenomenon at work in 3D visual represen-

tations that is not present in 2D representations where al1 information is restricted to

a plane perpendicular to the viewer. The addition of the thirci spatial variable leads

to the posçibility ofobjects interposeù (being positioned) between the viewpoint and

other objects in a scene, thus partiaily or cornpletely hiding them lrom a particular

view. The correct preservation of spatial relationships and presentation of occlusion

relationships is important in constmcting a scene with any degree of physical plau-

sibility; the development of accurate visible surface determination algorithms was an

active area relatively early on in the development of the field of computer gcaphics.

In using 3D representations in visudization, however, occlusion may work against

us. For example, in volumetric rendering of 3D data it is often the case that the

near-continuous nature of the data makes occlusion of interior features of the data

inevitable. This phenomenon is important in supporthg perception of tbe scene as

a 3D representation, but one may very weii wish to examine these hidden interior

regions.

Solutions are available to provide visuai access (clear lines of sight) to previously

occluded elements. Cutting planes rnay be used to remove information from a scene.

Increasing transparency (or reducing the opacity) of objects allows more distant o b

jects to be seen through those more proximal to the viewer. Navigation of the viewer,

whether egocentric (moving the viewer within the data space) or cxocentric (moving

or re-orientation of the data space) may lead to a contiguration where occlusion is

resolved. Finally, information Bltering rnay be used to d u c e the density of data

in a repmntation. These are al1 common methods of occlusion resolution and al1

operate by reducing the amount (or visibility) of contextual information in the fi-

nal presentation. Similar methods (such as panning zooming and filtering) have also

been traditionally applied to dealing with large or congesteci displays of information

in twedimensions.

1.4 Detail-In-Cont ext

The removal of information h m a presentation has been one approach to dealing

with the large information spaces. .4 second approach h a î been the development of

detail-in-context presentation algorithms.

The field of detail-in-context viewing is concerned with the generation of classes of

information presentations where areas or items defined as focal regions are presented

with an increased level of detail, without the removal of contextual information from

the original presentation. Early work in this ana includes the Bifocal Display of

Spence and Apperly [991 and the Fiheye views of Fhmas [39]. Each of these y-

tems sought to provide multiple regioas of scale within a single presentation. Fumas' subsequent Generalized Fisheye View [40] incorporateci the idea of a degcee of inter-

est (DOI) function. Given an objet or region of interest (ROI) the DO1 function

combined an a priori importance value for the remaining elements of the represen-

tation with their distance h m the ROI to determine their final importance. This

importance translated into the relative display size of the components. Thus regions

CHAPTER 1. 1NTRODUCTfON 7

of greatest interest were displayed at the largest size, providing more visual detail,

and the scale of the surroundhg context was adjusteci to provide the space for the

magdcation of the ROI. This may have been a simple scale adjustment or it may

also have involved symbolic replacement of elements where insufücient d e was avail-

able for a meaningful representation. Fumas' work included studies that pointed to

human information processing and organization operating in a manner that paral-

leled the properties of a fisheye display. People tend to retain and recd detailcd

knowledge about specific elernents of a domain and less detailed knowledge about the

extents of the domain. He suggested that the evidence from his studies indicated

that detail-in-context presentation methods may be an intuitive tool, leveraging this

human capacity for information navigation and retrievai in a manner not so dinerent

from the way in which 30 representations leverage Our spatial navigation and niemory

skills.

Recent detail-in-context mearch hss concentrated an the construction of multi-

scale presentation spaces for 2D information. Examples include: Perspective Wall 1721,

Document Lens [91], Continuous-Zoom [32,4], Rubber Sheet [94,95], Non-linear Mag-

nification [57], P d + + [5], Hyperbolic Space [65], CATCraph [55j, FocusLine [44] and

our own work on Three-Dimensional Pliable Surfaces (3DPS) [19], Work has also

been conducted in the area of detail-in-context views for 3D information representa-

tions including: Semnet (331, ConeTcees [92], 3DZoom [84], Visual Awess 1271 and

Non-linear Mapification [59].

1.5 Thesis Statement

A unique situation arises with detail-in-context viewing and 3D representations of

information, with the possibility of the ROI being occluded. As we have seen, there

are a number of methods which have been developed, independent of detail-in-context

viewing, for dealing with situation of occlusion in 3D information representations.

Each of these methods involves removing information hom a display in some manner,

which is contrary to the goals of detail-in-context viewing. This thesis presents a

new approach to occlusion cesolution that provides detail-in-context viewing for 3D

CHAPTER 1. INTRODUCTION 8

information representations. It maintains the representation of the data while deating

with occlusion where it occurs, namely dong the line of sight connecting the vicwpoint

to the ROI.

1.6 Thesis Outline

In chapter 2 we examine somc of the most important aspects and mechanisms of

human 3D perception as they apply to 3D visual representations of information in

visualization. We will aiso explore in more detail the field of visualization, speciB

cally the application of 3D visuai representations in a number of systems. Occlusion

reduction methods will be discussed and we will examine the field of detail-in-context

viewing for both 2D and 3D information spaces. The development of a novel viewer-

aligneci occlusion reducing transformation (ORT) operator, which seeks to integrate

the benefits of detail-in-context viewing with the occlusion reduction capability nec-

essary to deai with 3D information spaces is presented in Chapter 3. Chapter 4 will

demonstrate the application of ORTs to a range of 3D visuaiizations. Finally in

chapter 5 we diYcuss the potential for future work.

Chapter 2

Relat ed Work

In this thesis we are interested in the presentation and perception of three-dimensional

visuai representations of information on twdrnensional cornputer displays. Therc is

an immense amount of researcfi dedicated solely to the study of the psychophysical

and cognitive issues involved in 3D perception and we hope only to provide a broad

survey of some of the most relevant aspects of this field.

The 3D visual representations which interest us are particularly a product of the

field of visualization. A number of clrissification schemes dealing with aspects of the

visualization process have been proposeci. These analyses have mainly dealt with the

cognitive aspects of interaction with such representations and to a lesser degree the

visual structures employed. We will formulate a basic classification scheme that sorts

visual representations according to their spatial layout characteristics and use this

classification will frame our discussion 01 3D detail-in-context presentations.

We will begin with a brief examination of the mechanisms employed in the per-

ception of 3D scenes. Occlusion plays a significant role in our understanding of 3D

stnictures aud we will see how it relates to the other perceptual cues we may find in

an image.

CHAPTER 2. RELATED WORK

2.1 3D Perceptual Cues

The aspects of an image or sequence of images from which we derive information

about thc threc dimensionai structure of a scene are the depth cues within the image

or images. These cues are broken d o m into two priaciple groups; primary and sec-

ondary depth cues. The primary cues are those involving the operation of the human

vision system as it interacts with the 3D world. These include binocular disparity, as

weil as convergence and accommodation. The secondary depth cues are also called

pictorial depth cues. These cues are those which we cm be found in a 2D image and

do not involve the physical state of the human visual system to derive depth informa-

tion. Occlusion, motion parailax, kinetic depth effect, shading, sbadows, perspective

distortion, relative sizes of objets and changes in texture are al1 secondary depth

cucs. We have inchded a more detailed discussion of these perceptual mechanisms in

Appendix A. In producing images of 3D scenes in computer graphics we will apply some corn-

binatian of these primary and secondary cues in order to support the perception of a

scene as three-dimensional. Immersive virtuai environments (IVES) seek to produce a

sense of the user truly being a part of the 3D world. To that end they will cmploy as many of the primary depth cues as passible, especially stereopsis and motion paallax

with head-tracking. images in such NEs may be presented to users through the use

of head-mounted displays or surround-screen projections. In head-mounted displays

(HMDs) separate images may be presented to each eye to effect stereopsis, but since

the field of view in such systems is generaily narrow a second alternative of a single,

wider or wraparound, image is also an available configuration. These wider angle

single image displays seek to incorporate more of the peripheral vision system, as is

the goal of wraparound projection systems such as the CAVE [30]. In each of these

systems users arc generally head-tracked in order to provide the correct motion cues;

in HMDs the orientation of the viewer (the direction in which the user Ïs aiming the

display) is also tracked so that the correct view is presented as the direction of view

is rotated or tilted..

However, a problem arises in the use of stereo presentation in either of these sorts

CHAPTER 2. RELATED WORK 11

of systems. Since images are always produced on a single surface lacking the depth-of-

field of non-synthetic images, the cues of stereopsis, convergence and accommodation

will produce conflicting results as one's eyes h d only one distance at which to focus,

but encounter varying depth from stereopsis. This inconsistency is a major issue in

the fatiguing and disorienting nature of these environments.

More limited virtual realities are described as fish tank VR [Il41 or Desktop

VR [89]. In these systems a more limitai world is presented, typicdy a s m d volume

extending into and protruding out of a desktop (rather than head-mounted or sur-

round) display. in fish tank VR images may still be displayed in s tem, and the head

is tracked to produce appropriate motion parallax cues. Desktop VR is the display

of interactive 3D graphics on a desktop display without head tracking. DOOM and

similar garnes are examples of desktop VR environments. In each secondary depth

cues (perspective, shading, shadowing, texture, motion) play a strong role in inducing

a sense of emotional immersion in a scene.

In most cases we are likely to find ourselves limited to the situation of ûesktop

VR, lacking head-tracking and stereoscopic images. This means that a scnse of a

3D presentatiou space must be generated through the availability of secondary depth

cues. The lack of common access to primary depth cues in computer graphics is of

relatively limited concern in most tasks, since it ha% been shown that when applied

pmperly and in combination these secondary cues have a significant effect on the

3D perception of a scene and are only rnarginaiiy improved upon by thc addition of

primary cues [49, 1121.

Occlusion is the aspect of the display of information in three-dimensions with

which we are most concerned here. The correct presentation of occlusion relation-

ships, nearer elements in a presentation hiding those more distant from the viewpoint,

is a basic function in the creation of realistic computer-generated graphics. A consid-

erable portion of the early work in the development of the field of computer graphics

involveci techniques to derive the correct presentation of visible surfaces in the bai

image. The correct presentation of occlusion is a powerful tool in supporting the per-

ception of a scene in a 2D images as three-dimensional. Occlusion is such a powerful

secondary depth cue that it will override principle depth cues such as stereopsis in


situations where the two eues are conflicting. Occlusion is also a challenge in the use

of 3D computer-generatcd images in visuaiization. There is a sense in the field of

visualization that the application of 3D representations will incretwe the information

carrying capacity of a display, as well as leveraging the inherent human capacity for

comprehension within a 3D world.

.4t the same time the effect of occlusion is to limit the part of a representation

that is visible from any particular viewpoint. The work of Ware and Franck in [112]

examines the relative effectiveness of 3D displays, with levels of support for 3D per-

ception €rom desktop VR to fishtank VR, and fin& an increase in performance with

3D representations, though not directly proportional to the perceived increase in the

capacity of the information space. If we consider a 2D information space to have an

n2 information capacity (where n is the width and height of the plane), then the naive

expectation is that the addition of depth to the space will increase the capacity to

n3, a geometric increase in capacity. The work in [Il21 Ends a rather more modest

increase in eflectiveness of two to three times n2.

Three-Dimensional Visualization

The field of 3D information visualization has produced numerous examples of cep

resentations for abstract information. Concurrently much of the work in scientific

visualixation has centered around refining techniques celateci to the presentation of

volumetric data: such as surface extraction algorithms, direct volume rendering alg*

rithms and fiow visuaikation.

Numerous approaches have been taken to the construction of classification systems

with which to organize this space of 3D information graphies. These have included

examinations of the tasks related to the use of the visual representations of data,

studies of the stnict ural pmperties of the resulting information representations and

analyses of the cognitive aspects of the abstractions employed in information visual-

ization systems. We will examine a selection of these classifications of visualization

methods. We begin with a bnef overview of thefields of scientik and information

visualization.


2.2.1 Scientific Visualization

Scientific visualization is generally presented as a distinct sub-field, separate from

information visualization. The physical or simulatcd source of data in scientSc vi-

sualization (especidy in a subdomain such as medical visualization) often presents

an appropriate visual representation for the data, precluding the process of choosing,

or innovating, a new representation as we are often faced with doing in information

visualization. We are most interested in methods for the presentation of 3D represea-

tations, most notably for the presentation of volume data.

One of the simplest and most familiar 3D visualization techniques in scientific

visualization is are 3D surface plots. Surface plots are a simple extension of 2D plots

of functions or data to 3D with the addition of a third layout axis.

Figure 2.1: Image-order volume rendering proceeds across scan-lines, tracing tbe path of rays through the volume and performing shading calcuiations at regular intervals or at intersections witb the data grid.

More complex applications of visualization to scientific data include presentations

of information from fluid-flow measurement or simulation. A variety of methods have

been developed to aid in the visual representation of the paths of particles in a flow.

Figure 2.2: Data and normal values must be interpolateci €rom cell vertices to points within a cell. Three linear interpoiations are used to accornplish this; dong edges, then actoss faces, then through the cell.

Icons, m w s , or "hedgehogs" [103], particle animation [lM, 1081, streamlines or r i b

bons [109], streaklinm and line integral convolution (LIC) [14] are al1 common means

of pmducing a visual representation of the movement of particles owr time. Hedge-

hop and LIC are most often applied to 2D represeatations as their prcseutation in

3D leads to a degree of visual dutter which maktls the perception of spatial organi-

zation and structure more di@cult to interpret properly. Inten'ante and Grosh have

presented a number of techniques [SI, 521 which are designed to enhanee the correct

perception of 3D LIC structum through the application of color and shading as well

as by enharicing the appeararice of the depth-order of the elements of the diplay with

visibility-impeding halos.

In situations where experimental equipment or simulation produces scalar data

which is thtedimensional and apprmsimately continuous in nature this data is of-

ten described as volurnetric. V i d representation of such volume data may be a p

proactied in one of two principle mumeru, mainly depending on the character of the

data itself. Some volume data will contain distinct boundaries, sucb as between m w l e

Figure 2.3: Object-order volume rendering methods traverse the data set and determine the contribution of each element to the final image. Object-order methods may operate with front to back (Under operator) or back to front (Otrer operator) composition.

or other soft tissues and bone in 31) radiological imaging techniques such as Computed

Tomography (CT), Magnetic Resonance Tomography (MRT), or 3D Ultrasound. In these cases the boundaries in the data may be reprwnted as surfaces and surface

extraction algo&.hms such as Marching Cubes [70] (see appendix B) may be used to

convert these boundaries into geometric models. This geometry may then be treated

in the same way as ather 3D models, s a d offline, edited, rendered and interacted

with using traditionai 3D graphics techniques and accelierated hardware.

In other cases the data is more amorphous, lacking such distinct boundaries; for ex-

ample MEYT imaging of the brain. If surface extraction aigorithm are not feasible then

direct volume rendering (DVR) methods may be used to generate visual representations of the 3D data. DVR algorithms include ray-tracing of volume data [IO, 54,691,

splatting [118], shear-warp factorization [64, Fourierdomain volume rendering 1741

and hardware accelerated 3D texture rendering il, 13, 311. With the exception of

Fourierdomain volume rendering each of these algorithms operates by compositing


the values of the elements of the volume data (voxels or cells) which lie on the line of

sight (ray) behind each pixel.

Composition proceeds by adding up the values of the individual volume elements

encountered and summing their contributions accordiig to the intensity (value) and

opacity of the voxel, which is determineci by the application of a transfer function,

according to an operator such as the Over, Under or Maximum Intensity operators.

The process of composition may proceed Gom front-teback or back-tefront of the

data and may progress in imageorder (figure 2.1), pixel by pixel , as in ray-tracing

or in object order (figure 2.3), vaxel-plane by plane, as in splatting. Fourier domain

volume rendering can be described as a reversal of the data acquisition process. m i l e

Fourier domain volume rendering is able to quickiy generate views from new orienta-

tions of the data, the process loses the depth information in the images aud is not able

to produce visible-surface images as are the other DVR aigorithms. Rather Fourier

domain volume rendering produces X-ray like images of the accumulateci intensities

through the data. Here occlusion of internai elements is not as significant problem as

is the interpretation of the images to distinguish individuai components.

The remaining DVR algorithms are capable of producing realistic visible surface

renderings, or semi-transparent renderings where internai structures are reveaied by

reducing the opacity applied to specitic segments of the data. Maximum intensity

images of the data render only the brightest values encountered dong the ray travers-

ing the volume and are well suited to revcaiing internai structures which have been

artificially highlighted.

Another alternative to transparency as a means of revealing internai structures is

to apply cutting planes to remove information based on its location in the volume,

rather than removing it according to its classification with reduced opacity. Note

that each of these solutions (cutting planes and transparency) reveal elements of the

volume by removal (partiai or complete) of occluding elements.

A more recent approach for providing internal access to regions of a volume

rendered image comes from the work of Kunion and Yagel on Discontinuous Ray-

Defiectoa [61, 621. We examine the operation of ray deflecton in more detaii in section 2.5.1 of this chapter.

CWTER 2. RE:LATED WORK

2.2.2 Information Visualization

In moving towards the use of tbree spatial dimensions in information visudization we

expand the repertoire of spatial and visual variables available to us in the construction

of a representation. The addition of a third spatial variable creates a volume in which

to arrange elements rat her than the line or plane we were restricted to in using one or

two dimensions for layouts. A n understanding of the differences in the perception of

volumes versus areas is important in the formulation of representations of information

where volume is used to encode information. Me [103, 104, 1051 provides many of

examples of the misrepresentation of quantities in visual displays of information that

may arise from the careles application of three-dimensional features to an information

reprmntation.

The collective space in which we represent and experience visual abstractions of

information has corne to be known commonly as "cyberspace". This t e m was coiiied

in a 1984 novel, Neuromancer, by William Gibson. ' Cybenpace has also been t e d Benediktine Space, after the work of Michml Benedikt who described the

characteristics and principles of such a space [6]. In describing this space, Benedikt

identified the intrinsic and extrinsic spatial dimensions of componcnts arrangeci in the

space. The extrinsic dimensions of an object specify a point within s w e , while the

intrinsic dimensions specifj the object's attributes: color, shape, texture and size.

This formulation of intrinsic and extrinsic variables ia similar to the spatial and visual

variables identified by Bertin in [8]. Benedikt also describes a number of principles

for the distribution of elements reprcsenting abstractions of information within such

a space.

"Cybenpace. A consensual bducination experiend d d y by biiiione of legitimate operators, io every nation .... A graphie representatiw of data a- fmm the banks of every cornputer in the buman system. Unthinkable complexity. Lines of light ranged in the nonspace of the mind, clusters and constellations of data Like city Jigbts, teceding ..." - Wilüam Gibson, Pieuromancer, 1984


The realm of information visualization bas produced a host of approaches to the vi-

sud tcprescntation of abstract forms of information. In order to structure and analyse

these methods, as well as provide a basis for future approoches, several classification

shemes have been proposeci.

Cognitive and Structural Frameworks

Wiss and Carr examine the cognitive features of a number of information visualiza-

tion systems [121]. The specific cognitive aspects considered are: those conmming

the methods employed to draw attention to significant elements of the data, the

means of supporting information structuring and information hiding through abstrac-

tion and the affordances which the system offers users as a means of interaction.

in analysing the cognitive aspects of information visualization systems the authors

present a second, structurai, framework for the classification of such tools. The

four broad categories of designs identified are: node-link styles (SemNet [33], Cone

and Cam 'ikees [92], Hyperbolic Space [65] and SeeNet3D [28]), raised surface styles

(Perspective Wail [72], Document Lens [91] and 3DPS [19]), information landscapes

(File System Navigator, Harmony 121, SDM [26], Bead [25] and WebForager [17]) and

"ot hern designs ( WebBook [17], Information Cube 1861 and n-Vision [Xi]). While

many information visualization systems support interaction with the user via control

panels, the authors stress their belief in the importance of direct manipulation; their

analysis of the cognitive affordances of interface designs is approached with this in

mind. While the focus of their work is an analysis of the cognitive issues , the structural

classification is equally intcresting to our own work. The noddink style and "other"

categories include information visualization tools in which the layout of information

is three-dimensional. The raised-surface family of interfaces is applicable to informa-

tion which is principally linear (in the case of Perspective Wall) or plmur in nature.

While these are 3D information visualization tools, the third dimension hem fulfiil a

role in attribute emphasis (detail-in-context viewing) rather than as a spatial aspect


attributed to the data itself. The information landscape group of interfaces repre-

sents visualization systems where the organization of data is principaily 2D but the

of individual entities are abstracted as 3D representations of objects on a landscape.

in the cases of the raised-surface and landscape interfaces the restricted layout

means we can readily find viewpoints which minimize the effects of occlusion. How-

ever, the 3D layouts of node-link designs such as SemNet [33] and "other" designs

such as the Information Cube [86] imply that occlusion rnay be a problem for which

some solution other than choice of viewpoint is required.

Other schemes for the classification of information visualization methods have a p

proached the process from the analysis of the tasks performed while interacting with

the visualization systems. Ctud [16] identifies four functiond levels in the process of

"information perceptualizationn . These four levels are: the infosphere, the workspace,

sense making tools and the document. Approaching the description of a visualiza-

tion tool from this perspective allows Card to separate the function of the tool from

the technique itself, as a speciûc technique may be applied across scvcral of these

functioual levelu.

The infosphere in this scheme is the space of al1 available information sources,

such as databases and documents. Tools for the visualization of the infosphere are

capable of providing overviews of this space and perhaps incorporating a semantic or

spatiai organization of the resulting structure. Narcissus [46], Hyperbolic Space [651

and %orlds within worlds" [35] are al1 presented as examples of infosphere-level vi-

sualization tools.

The workspace level actions involve interactions with groups of objects that have

been arrangecl in such a manner as to make the completion of certain tasks more

efficient. The role of visualization tools in this situation is to improve the efficiency

of interaction with the structure of the workspace. This can be accomplished through

the utilization of faster perceptual or pre-cognitive attributes of objects rather than

cognitive properties and by increasing the information carrying capacity of the display


as a whole. The use of zooming interfaces in workspaces such as Pd++ [5] and multi-

d e interfaces such as Bifocal-Diplay [99] to provide detail-in context presentations

are other means of making interaction with the workspace more efficient.

Sense making-level tools are those which assist in the understanding of information

through the creation of combinations and associations. These tools may be static

or dynamic in presentation but are intended to reveal patterns in information. For

example, the cone tree [92] turns part of the WWW into a tree according to a traversal

algorithm, and the Table Lens system [85] presents a detail-in-context display, which

supports interaction with the rom and columns of a worksheet to reveal patterns in

the data.

Finally, document-level tools interact with the elementary units of information

retrieved. Wi thin an individual document the contained information itself may be

large and poses interesting interna1 structure.

Shneiderman [98] formulates a taxonomy incorporating both the data type and the

proposeci task. Where Card presented four task levels, Shaeiderman provides seven

more specific task descriptions: overview, zoom, filter, detailson-demand, relate, hi+

tory, extract. The examination of the relations between these two schema provides

a richer illustration of the process of interacting with an information space. In the

infosphere we wish to have an overview of the entire information space. We zoom

and filter this space in order to construct workspaces. Within these workspaces we

d a t e information elements to discover patterns in the process of wnsr! making, and

by maintaining a history of actions we may retain previous patterns to aid us in the

construction and discovery of newer ones. Finally we want to be able to extract details

and subgroups to support document level investigations and abstractions. Shneider-

man also presents his information seekng mantm "OveMew first, zoom and filter,

t hen details-on-demand."

Shneiderman also identifies seven data types in the classification of information vi-

sualization systems: one-, t~+, three-dimensional, temporal, multi-dimensional, trees

and networks. Shneiderman recognizes that many information vhalization systems

are oriented towacds dealing with a specific class of data, for instance: Geographical

Information Systems with 2iiimensional data, LifeLines [76] and Perspective Wali [72]

CHAPTER 2. RELATED WORK 2 1

with temporal data, Cone and Cam Thes [92] for tree data and SemNet [33] and te-

latcd tools for network data. The author beleives that a t d y successful information

visudization systems will have to be designd to accommodate severai classes of in-

formation and the full range of taaks simultaneously.

2.3 Structural Framework for Visualization Design

We propose a simple classification system for 3D visualization systems, whether sci-

entific or information, b d principally on the characteristics of the resulting visual

representations, rather than on the characteristics of the data. In this respect our

classitication is somewhat more similar to that outlined by Wiss and Carr [121] than

that of Shneideman [98]. We identify three principle characterizations of visual reg

resentations: discrete, contiguous and continuous. These three choices are motivated

to some degree by the specific challenges they pose in the application of OUI occlusion

cesolution tools, which we will examine in further detail in Chapter 4. We would

describc the characteristics of cach class of reprcsentation as follows.

(a) D i e (b) Contiguous (c) Continwus

Figure 2.4: Structural classes of threedimensional representations

Discrete information layouts are exemplifieci by node and edge structures or 3D

scatter-plot layouts. Information representations of this class are characterized as

having spatial ordering relationships where connections that exist within the structure

are represented by explicit connectivity (edges) rather than physical adjacency.

Contiguous information representations, include 3D models, finite element sets,

CAD data and so on. In these models not only is spatial ordering important but

so are the physical properties of adjacency and containment. Distortions which are

applied to this class of information representations may benefit from a treatment

that accouats for coilision detection. Performing Layout adjustments which resul t in

components translating through each other would otherwise violate the perception of

the components as comprising solid surfaces.

Continuous representations may be truly continuous, as the product of 3D para-

metnc equations producing a volumetric lunction, or the may be such hely diiretized

data as to appear continuous in some sense, such as volumetric medical imaging,

geophysicai or fluiddynamics data. These kinds of data are generally tackled the

approaches describeci in section 2.2.1

2.4 Methods for Occlusion Reduction

In previous sections we bave briefly discussed the approaches that various visualization

systems have used to reduce the occlusion of elements within a representation. In the

following discussion we will present these traditional approaches in more detail and

discuss their operation independent of specific visualization tools.

2.4.1 Navigation

In the case of relatively sparse layouts of 3D information, movement of the viewpoint

is a cornmon solution to the situation of nearer objects occluding more distant ones

of potential interest. The ability to move the viewpoint or re-orient a model is a com-

mon md now expected feature in any system which employs interactive 3D graphies.

Beyond offering users the opportunity to ünd solutions to instances of occlusion, the

ability to produce movement is a powerful means of enhancing the perception of a 3D

structure. Figures 2.5(a) and 2.5(b) illustrate two different views of a skeletal model


of a human foot. In figure 2.5(a) the first metatarsai bone (which is highlighted) is

hidden from the current viewpoint through occlusion by other bones in the model. In figure 2.5(b) the viewpoint has been rotated about the center of the model in order to

move to a new position from which the view of the metatard is no longer occluded.

(a) ûcciudcd Vicw (b) Non-accluded view

Figure 2.5: Moving the viewpoint makes the highlighted bone (the first metatarsal) occluded in (a) visible in (b).

Tùis movement is geueraiiy passive movement of the data space, through input

with a mouse or similar device. The change of VP may also be accomplished by active

movement of the viewer, along with appropriate tracking technology to automatically

update the view. In the passive rnovernent approach the metaphor of a turntable is

often employed. Movement of the VP is about a centrai fixed point of reference and

the view remains directed at this point. Commonly the movement of the data dards

complete cevolution about the vertical (y) axis and Iimited movement of the VP up

and down around the horizontal (x) ais. Some means of zoorning the view in and

out from the point of reference is also common. Generaiiy zooming is accomplished

by the movement of the VP towwds or away from the center of the data set, rather

thaa by means of narrowing of the field of view wbich is tbe mare common definition

of zoomhg in optical terrns. The correct tenn for the movement of the viewpoint in

and out in this manner is dollying. Such movement of the viewpoint into an out of

a structure does offer a simple means of rnoving past occluding objects to produce


a clear view of those previously obecured. In figure 2.6(a) the node in the center of

the g3 element 3D-graph is highlighted but aimmt entirely occluded by other nodes

in the structure. Here movement of the VP to another extemal viewpoint is unlikely

to provide a resolution of this occlusion. in figure 2.6(b) the viewpoint has been

moved into the stmcture, past the occluding elements to provide a clear view of the

highlighted central node. The side effect bere is the loss of much of the structure from

the display as it is outside the field of view or now behind the viewpoint.

(a) Externai Viewpoint (b) internai Navigation

Figure 2.6: Movement of the viewpoint into the structure puts elements that occliided the node of interest in (a) behind the viewer in (b).

2.4.2 Partial Tkansparency

A phenomenon closely related to occlusion is the partial occlusion produced by semi-

transparent or sik surfaces. These surfaces do not completely obscure more distant

objects and have been widely used in cornputer graphics and visualization to pro-

vide a sense of solid 3D structure with minimal lm of information through ocdusion.

Cone Trees [92] and Spiral Cdendar [73] each applied partial occlusion to improve the

appearance of the spatial structure of the viçual representation. Partially transparent


cutting planes have been applied in surgical visualization tasks to facilitate the ori-

entation of the plane with the organs intersected [i24]. Such see-through surfaces are

also seen in 2D information spaces as a means of creating a 2.5D space for example

as in Tool Glasses [9] and Silk Cursors 1122, 1231. The work of Zhai et ai. on 3D silk

cursors [124] indicates the effectiveness of partially transparent curson in six degree

of f'eedom (6DOF) tracking tasks. Silk cursors were found to be more effective than

wire-frame curson in 3D locaiization tasks in both mono and stcrm viewing.

Figure 2.7: Partial Tkansparency

In task domains with increasingly cornplex surface topologies partial transparency

increases the difficulty of perceiving the shape of the surface and makes the distinc-

tion of multiply layered surfaces more difficult. For example in figure 2.7 partial

transparency makes the interpretation of the details of the distinct ,layececi, surfaces more chailenging. The work of Intecrante et al. in [50, 531 examines the application

of contour-driven textures to improve the comprehension of such structures, at the

expense of increasing opacity.

Cutting planes and volumes have long been a standard feature of direct volume ren-

dering systems. Cutting planes are a highly effetive means of providing visibility of


voxels adjacent to a plane through a volumetnc data set or at a specific location within

a less dense representation of direte components or surfaces. Cutting volumes are

used to remove more cornplex regions €rom a display, rather than entire half spaces.

These volumes define regions that are removed in a mariner similar to constmctive

solid geometry (CSG) subtract operatioos. Figures 2.8(a) and 2.8(b) illustrate the

effect of a cutting plane and a cutting volume on a volumetric data set. Cutting

planes and volumes have the effect of remedying the occlusion of areas or regions

of interest at the expense of removing other information from the final presentation.

There are additional costs in terrns of the complexity involved in the specification

of the placement and orientation of cutting planes encountered in practice. Many methods have been examined to adàress this problem of specification of cutting plane

placement including twehanded interactioas wi th prop to simplify these tasks. [48]

(a) Cutting Plane (b) Cutting Volume

Figure 2.8: Cutting planes and regions remove volumetric data in a half-space (a) or subvolume (b) from the bal image and make previously occluded surfaces visible adjacent to the cut.

Segment removal from direct volume rendering is another method for the reduc-

tion of occlusion. This is analogous in 3D information visualization representations

to the application of a Dû1 function to Mter elements out of the final presentation.


Figure 2.9: Selective removal of component groups improves visibility of remaining components.

in volume visualization the application of transfer functions determines the color and

transparency of a specific component such as s k i , bone, muscle, or other intemal

organs. Individual component alpha values may be adjusted (lowered) so as to re-

move component elements and reduce the effect of occlusion on thase remaining. In

figure 2.9 we have reduced the opacity of the outermost layer in the structure to zero

in order to achieve a cleorcr view of the rernoining two internai components.

2.5 3D Deformation Methods

Methods for the defortnation of 3D models are intrinsic tools in the production of

interactive coniputer graphics. Deformation of models may be in the context of a

simulation of a physical process, in producing animation for film or television or in

interactive graphics for computer games. Many techniques have been developed and

applied acrm a range of 2D and 3D graphical structures. OveMews of this field

and the details of many of these techniques c m be found in most computcr graphics

textbooks, notably [36,41, 1161.

There is a comparatively small set of methods for the transformation of graphitai

objects which are of speciüc significance in their relationship to the techniques which


we will develop through this work. The most significant systems are the Discontinuous

Ray Deflecton of Kunion and Yagel [62] the Zoom Illustrator work of Preim et

al. [82, 83, 871 and the Page Avoidance component of the Data Mountain system

developed at Microsoft by Robertson et al. [88].

2.5.1 Space Deformation Operators

Space deformation operators [61, 62, 631 provide both a mechanism for warping of

3D volumetric data or models and, with the addition of discontinuous deflecton [62],

it is possible to arrange tbese operators in su& a manner as to provide visual access

to partial cut-planes through volumetric data sets. The operation of ray deflectors

is Erst described in [61]. A ray deflector causes a locally constrained deviation in

the path of a sampling ray and results in the apparent couaterdisplacement of the

sampled surfaces (figure 2.10).

Figure 2.10: The effect of a warp operator on the path of a ray through a scene. The deflected ray results in the appearance of a deformation of the surface. (After [61])

Discontinuous ray deflectors restrict a ray to sampling on only one side of a plane.

The r d t is that the outer surface of a volume is split and the data on that plane is

made visible (figure 2.11) as the swrounding material is pushed aside.

The application of ray deflector operaton to volume rendering with hardware

assisteci volume rendering is examineci in 1631. The process of applying hardware

texture mapping accelerators to the process of volume rendering is described in [l].


Figure 2.1 1: Discontinuous ray deflecton operate by deflecting rays in opposite directions h m opposite sides of a plane. Ray sampling is restricted to the original side of the plane, thereby producing a cutting and retracting form of distortion. (After (631)

Kunion and Yagel apply the inverse of the ray-deflection method to deform points

in tessellated planes, thereby performing a piecewise linear approximation of the ray-

deflection operation. The application of Discontinuous deflectors in this context leads

to the problem of splitting the tessellateci planes according to sampled and un-sampled

vertices.

2.5.2 Zoom Iilustrator

Zoom illustrator [82, 831 extends the continuous zoom algorithm [4, 321 h m two di-

mensions in order to apply it to interaction with three-dimensional models of anatom-

ical structures. The effect of the zoom algorithm is to emphasize the appearance of

objects of interest within the model by applying magnification to these elemcnts.

In order to facilitate this magnification with the original space of the 3D layout the

surroundmg elements of the model are reduced in d e accordingly to provide the nec-

essary space. The interaction with such anatomical models as 3D puzzles is explored

in the work of Ritter et al. [87] in which a variety of 3D interaction and presenta-

tion strategies are explored. These techniques include the zoom algorithm as weii as

the use of partial transparency and shadows to enhance the perception of the spatial

rdationships of the 3D elements.

2.5.3 Page Avoidance

Data Mountain, developed by Robertson et al. [88], is a 3D document management

system. The system was initially applied to the spatial arrangement of and interaction

with Favorites or Bookmarked pages from a web browser, and has since been incor-

porated into the tool palette of the Task Gallery system [go]. The Data Mountain

allows for the movement of these pages, visually represented as textured images of

the actual web page on rectangular polygons which remain perpendicular to the view

direction.

An important part of the Data Mountain environment is the incorporation of a "page avoidance" behavior exhibited by the individual page elements. Each page

maintains a minimum distance from al1 other pages in order to prevent one page fiom

completely occluding another. The movement of one page by the user results in other

pages moving out of its way. The movement of pages is propagated to similarly avoid

occlusion of other pages. The Data Mountain envitonment in two inctrrnations (DM1 and DM2) is compared in a user study with Microsoft Internet Explorer 4.0 (IE4). The

differences between DM1 and DM2 include the addition in DM2 of page avoidance,

stronger association of "hover titlesn with pages, and the addition of spatialization

effects to the accompanying audio feedback. The k t significant result of this study

to our work is that the DM2 users showed reliably as fast or faster retrieval times

than the IE4 or DM1 users, with fewer incorrect cetrievals. The second significant

aspect is that in a subjective survey usen expressed a preference for the DM2 system

wer IE4 while they did not prefer DM1 over 1E4.

The page avoidance algorithm of Data Mountain proves to have a strong resem-

blance to the occlusion avoidance algorithm we will develop in this thesis when applied

to sirnilar discrete representation. We will see a more detailed discussion of this a p

plication in Section 4.1.4

CHGPTER 2. RELATED WORK

2.6 Det ail-in- Cont ext Viewing

Detail-in-context viewing for 2D information presentation bas a history going back as far as 1981 with a Bell Labs technical report by h a s [39] which introduced the

notion of a "fish-eye" transform of a document and an associated degree of interest

function. S u b u e n t work, notably by Spence and Apperly [99], extends the appli-

cation of these "fish-eyen views to graph layout and later to general raster images by

Carpendale et al. [19, 201.

2.6.1 Perspective-Based Fisheyes for 2D layouts

A 3D metaphor for the creation of detail-in-context views of 2D information repre-

sentations was first demonstrated in the Perspective Wall system developed at Xerox

Parc by Card et ai. The Perspective Wall and derivative systems generate magnifi-

cation and compression of layouts by manipulating a 2D surface in three dimensions.

Pulling region of the surface up towards the viewpoint, in conjunction with the effect

of perspective distortion makes that part of the surface appear larger. Perspective

Wall is designeci to work with data representations that are 2-dimensionai but have a

dominant horizontal dimension such as: tirnelines, digital video, and visual represen-

tations of digital audio.

The data is arranged on the wall and a detail-in-context view is obtained by pulling

a section of this wall towards the viewpoint in a 3D perspective viewing projection

(as observed by [33] an orthographie projection muld not produce the same effect).

The section of the wall which is puiied towards the viewer remains perpeodicular

to the line of sight. Were it not to do so apparent magniikation across its surface

would not remain constant as the magnified area would cover a range in z (depth)

dues . The remaining segments of the original wall remain attached at the left and

right sidei of the magnified region and appear to recede away fiom the viewer to

the original depth of the wall. These bracketing regions are therefore at an oblique

angle to the line of sight and the apparent magnification at any point on the d l in

the contextual region depends directly on the depth in z of that point and inversely

on the angle between its normal and the line of sight to that point. Figure 2.12(a)

CHAPTER 2. REtiATED WORK

(a) No R@on of Intu& (b) Flat Surfacc

Figure 2.12: The base configuration of the Perspective Wall with no regions of interest (a). The surface has one dominant linear dimension and is at a constant depth in z in the perspective viewing frustum (b).

illustrates teb perspective wall in a neutral state with na focal region, figure 2.12(b)

shows the relative configuration of the perspective wall surface, the viewpoint and

the perspective view volume. In figure 2.13 a region in the center of the penpcctive

w d l is selected as a region of interest and puiled up towards the viewpoint in order

to produce a detail-in-context view.

The Perspective Wall was soon followed by a related system frorn Xerox PARC, the Document Lens [91]. In the Document Lens the data space is more traditiondy

2D, and does not assume a principle, dominant dimension in which the data is mainly

linear. The document lem is so aamed as the initiai applicatim of the system was

for browsing a document corpus and the focal region ia generally defined as a single

page in the set. In this system there are contextual regions to the top and bottom of

the focal region as well as to its left and ri&. The distorted surface appears to take

on the shape of a tnuicated pyramid. The document of interest forms the top of the

pyramid and the sunounding, contextual documents Form the sides of the pyramid.

The distortion is constrained to the extents of the original (0at or undistorted) layout,

hence one or more sides of the pyramid may take on a more distorted appearance than

(a) Central ROI (b) ROI Pulled Up

Figure 2.13: Perspective Wall with a single ROI specified in the middle of the field of view, generating a region of increased scale and surroundhg distorted regions (a). The ROI is at a constant depth in z with respect to the viewpoint in the perspective viewing frusturn (b).

the others as the focal area is rnoved towards or away h m a particular boundary.

Both Perspective Wall and Document Lens engage human perceptual abilities in

interpreting the 3D metaphor of perspective distortion. The goal is to support corn-

prehension of the effect of distortion applied in order to obtain the detail-in-context

view. The use of the perspective view-volume introduces sorne particular constraints

on the mannet in which the surface may be distorted. Figure 2.14 illustrates the fea-

tures of the perspective viewing frustum. The pempectivc view volume in cornputer

graphics fonns a pyramid. The boundary of the pyramid is defined at its vertex by

the viewpoint and at its base by the far plane. The pyramid may have a square base

but more generally it has a base which is a rectangle, the relative dimensions of which

are defineci by the width and beight of the viewing window; the ratio of the width

to the height defines the aspect ratio. In the traditional cornputer graphies pipeline

the perspective projection transformation is followed by a viewport to screen trans-

formation which may change the relative width and height of the image, hence the

aspect ratio of the ha1 rendered image may not be identical to that of the perspective


view-volume. In fact if a canonical perspective view volume is defined to aid in the

process of 3D clipping, then it wiii have a square base (aspect ratio of 1) and the

viewport transformation will restore the appropriate aspect ratio.

Figure 2.14: The features of the perspective viewing frustum. The frustum Forms a pyramid with the viewpoint at the apex. The far-plane foms the base of the pyramid. The widt h and height of the pyramid are usually defined by the field of view and the aspect ratio. The field of view is the anguiar horizontal width of the pyramid, and the aspect ratio defines the relationship between the width and the height. (a = -1. Objects within the pyramid arc visible in perspective projection if they am located between the near and far planes in depth. The central axis of the pyramid defuies the direction of the view in world coordinates.

The geometry of the pyramid imposes some restrictions on where a surface element

may be moved to and remain visible in the final image. Notably, an element lying on

the line of sight (a line vertically through the center of the pyramid) d l remain in the

center of the field of view as it moves dong this line of sight and perpendicular to the


base of the pyramid, towards or away from the viewpoint. Conversely an element that

lies to one side of this line of sight, also moving perpendicular to the base (translation

in z only with no change of x or y coordinates) will appear to move away from the

center of the field of view as it moves towards the viewpoint. Thus a focal region in

Perspective Wall or Document Lens that lies in the center of the field of view requins

no special consideration. Should the region be centered anywhere else then moving it

in z to produce magnification will have the undesirable effect of an induced translation

which will move the region out of the field of view. The solution to this problem, both

in Perspective Wall and the Document Lens, was to shear the view volume such that

the viewpoint was moved directly over the center of the focal region as illustrated in

figure 2.15. Regions may now be translated simply in z and remain within the field

of view. This choice of viewpoint movement does introduce a further restriction ou

the range of views which may be constnicted. As the perspective viewing volume

has only one viewpoint the requirement that it be positioned directly over any focal

regions thereby restricts the system to a single focus.

(a) On-Center ROI (b) 'Ltanslation of VP in z

Figure 2.15: Geometry of the fnistum is sheared in x to keep the Viewpoint directly over the offcenter ROI.

Multi-focal distortion viewing via the effect of perspective distortion was intro-

duced in [19] with Thcee-dimensional Pliable Surfaces (3DPS), which later became

known as an example of an Elastic Presentation Space (EPS) [23]. 3DPS implements

a system by which regions of interest are translated not in 2-only, perpendicular to the

base of the perspective viewing volume, but dong sigbt-iine-aligned vectors. The use

of sigbt-line aligned distortions provides for the speci6cation of multiple focal regions

within an information pmentation space and the process of blending affords a means

of controlling the interaction of these multiple regions.

(a) Vertical Mavernent

Figure 2.16: Effect of simple vertical movement of a portion of the surface in an ofl-ccnter lem alter perspective projection (a). The rcason is that the surface now extends outside of the perspective viewing Gustum (b) .

Figure 2.16(a) is an example of a 3DPS surface with a focal region a t the right

edge of the plane, with the region pulled up perpendicular to the presentation plane.

Figure 2.16(b) is a side view of tbis situation illustrating the way the focal region

moves outside of the viewing fnistum. The solution in 3DPS to this situation was to

shear the focal region back t o d the viewpoint rather than shearing the viewing

h t s u m itself as we see in figure 2.17. This, dong with a method for mediatiag the

interaction of focal regions thmugh blending, allowe for the sirnultaneous specification

of multiple focal regions as in figure 2.18. The construction and application of these

distortion views are dmribed in detail in [19, 221 but WU be revisited in Section 2.6.1.


(a) Corrected Movement (b) Viewing h t u r n

Figure 2.17: Shearing the distorted rcgion so that it is orient4 towards the viewpoint (a) brings the entire extent of the lem back into the projected image(b).

Mat hemat ical Framework for Layout Adjust ment

We begin from the observation that the integration of the region of intetest, the

surrounàing contextual region of compression and the region of the presentation space

which remains undistorted may be determined by the specification of the profile of

a crosssectional curve. This curve is used to join the region of interest, which has

been pulled towards the viewpoint and is at some height h with respect to the original

layout, with the plane of the original layout. The characteristics of this curve will

determine the distribution of compression within the contextual regions and the nature

of the connection of this region to the focal and original layout areas. Perspective

w d and Document Lens employed a linear segment to connect the focal region to the

original plane of the image. (Perspective Wall and Document Lens both distributecl

distortion hom the edges of the focal region to the extents of the presentation space,

subsequently leaving no undistorted regions)

The mathematics governing the appearance of these distortions is examineci in [67].

While the context of that anaiysis is primarily 2 0 + 2 0 translormations the use of

(a) Multiple OtI-Center ROI (b) Viewing Fhistum

Figure 2.18: Shearing the lenscs rather thm the viewing h t u m allows for the specification of multiple ROI (a). The area of intersection of lenses must be blended to provide a smooth transition between the two shearing directions (b).

2 0 + 3 0 + 2 0 transformations is simply another framework for the conceptual-

ization of the mathematical operations. The effect of the operations in the 6nal 2D

projection are the sarne.

if we begin with the Gaussian curve as the cniss sectional profile f(x), as in

equation 2.1 and figure 2.19, the c m has a maximum height of 1. If we define a

simple perspective view-volume with a viewpoint at a distance of 2 from the original

Figure 2.19 The Gaussian curve j(x) = e-lo-ot' used in ThreDimensional Pliahle Surfaces to provide smooth integration of the ROIS and original information layout.

Figure 2.20: After perspective projection, the apparent transformation t (x ) of points on a suriace traasformed by the application of a garnian lem with a maximum height of 1 and a viewpoint distance of 2 from the original surface plane.

plane of the presentation surface, then we c m determine the resdting translation t ( x )

of a point on this c m after perspective projection, as in equation 2.2, expandeci in

equatioa 2.4 and ptotted in figure 2.20.

zL-lo.Os~ Figure 2.21: The displaced putition of points d(x) = x + ' l - C - L O , O = ~ as a regult of the gaussian lens after perspective projection.

This produces a displaced layout d(s) as in equations 2.5 and 2.6 and plotted in

figure 2.21. If we removed the effect of the transformation we would have d(x) = z.

Finally we determine the relative magni6cation or compression, m(x), of a region on

this line as the derivative of d(x) with respect to x, as in equations 2.7 and 2.8 and

figure 2.22.

Table 2.1 illustrates the relation between the shape of the 3D displacement curvc

profle, the resulting apparent displacement of points in perspective projection.

Figure 2.22: The magaification (with singlepoint perspective projection) and compression distribution as a mit d the gaunùan l e m m(z) = y

Figure 2.23: The progression of Lp distance metrics £rom L1 (figure 2.23(a)) to LSOO (Bgure 2.23(d)) in two dimensions.

Anotber aspect of the specification of a transformation hnction in a system such as

3DPS or EPS is the measurement of distance from a point to the nearest focal points.

Carpendale describes the use of a distance metric other than simple Euclidean distance to d k t the shape of focal regions in [23]. The Lp distance metric provides a means


Hemi. ] Coeine Tan 1 Inv Hemi.

Table 2.1: A visual cornparison of a range of magnification functions (Constant J(x) = -0

1, Linear f (3) = 1 - 2, Gaussian P ( x ) = e., Hemisphere J ( x ) = sin(cosh(1 - x)), ~o(O.Bs*m ' ) Cosine f (3) = ms(z t f ), Tangent f (z) = 1 - and Inverse Hemisphere

f (z) = 1 - sin(eosh(1- 2))) and their pmperties of slope f , apparent planar translation t (z) and resulting m@cation m(x) (within a perspective-distortion system such as 3DPS).

of varying the appearance of a focal region, varying the shape of its boundary from dittmond-like to rectangular by adjuating the value of the p parameter in equation 2.9.

Figure 2.23 illustrates the effect of varying the value of p h m a minimum of 1 in figure 2.23(aj to a maximum of 200 in figure 2.23(d).


Magniacation versus Displacement

In [18] the relative roles of magnification and displacement in 2D detail-in-context

viewing are cxplored. We observe that the movement of elements dong the h e

of sight for a given focal region produces a magnification effect due to perspective

distortion.

We further note that in the case of a discrete 2D graph the is that locally dense

layouts of edges become l e s dense at the expense of some compression at some other

location. Since it is possible to separate the action of displacement h m the mng-

nification of nodes in such layouts local layout adjustments may be applied to the

problem of "cluster-busting" in dense regions of graphs. This capability holds true

in 2D and 3D information layouts and has some utility in reducing tbe problem of

occlusion in locally dense regions as illustrated by Keahey with layout adjustment of

3D structures in (591

Chapter 3

Met hod

Wt! have identifid in previouri chaptem that there hm ben a great deal of work in

the creation of 3D visual representations in visualization. At the same time there has

developed a strong interest in the ddail-adcontext presentation of information in

both 2D and 3D representations. Touls for the generating such preseatatioas of 2D

data have beeri the principle kw of the mrk in t h m a . The mite of tooh aMiluble

for the generrrtian of cietail-and-contrrrt view of 3D represeatationa of information is

miich ~mdler thm thme which operate on 2D data

One of the principle differences that we face in dealing with 3D representations

is thc issuc of occlusion. Whcn dcaüng with 2D displays of 2D rcprcscntations WC

do not have to concvm our~elvev with elementu of the information Iayout becoming

hidden behind other elements. Adding a z component to the layout space intmduces the possihility that some elemeniq wiii hidden. F&ensiona of clawicai detd-

and-mntext viewiag algorithm to 3D through the addition of a z component d m

not dcquatcly a d h this situation.

We cm homver mtend the application of mme of t h e 2D twhiqum to 3D in a mamer that does account for the presence of elements occluding the object of

interest. We wiii accomplith this hy d n i n g the proces8 of generating 2D detail- and-context views and identifying the specific elementa of the transformation process

which contribute to ducbion ia local idmation density the hiil llryout.

3.1 Layout Adjustment in 2D

Our method for occlmion duction and detail-incantext viewing of 3D cepremta-

tions has p w n out of our previous work on the 3-Dimensional Piable Surfm sys-

tem (JDPS). 3DPS created detail-in-context views of 2D visual repreaentations with sight-linedigned distortions of a 2D information preaentation surface within a 3D

penipective viewing fnistum. In 3DPS magnification of regions of intereut and the riccompying compremion

of tht! mntextiiai mgion to accammodate thia change in utde are prodiiced hy the

movement of regions of the surface towards the viewpoint (VP). The process of p m

jccting thcsc transformcd layouts via a pcrspcctivc projection miultcà in a ncw 2D Iiiryout whicb included the zoomed and c u m p d regions.

The use of the third dimension and perspective distortion to provide magnification

in 9DPS pravidm a meaningful metrphor for the pmem of diatorting the informrtion

presentation surface. The 3D manipulation of the information presentation surface in sucb a systcm is an intcrmcdiatc stcp in thc proccss of crcating a ncw 2D layout of thc

information. In wction 2.6.1 we saw that a ttsnsformation function from 2 0 + 2 0 is possible, if we incorporate the effect of the perspective projection on the layout

adjustment function.

If we concentrate on the 2 0 + 2 0 translation function t(s) we can apply this to

d u c e the local density of elements in a layout as demonstrated in (181. Thia effect

of local density miuctinn is significant, as is the ahiity to separate the trandation

component of the lem h m the magnification fimaion when dealiag with dimete

structures. It is precisely this ability to reduce, or rather distribute, the density

of information in a rcpmntation that WC apply to thc problcm of occlusion in 3D repmntationx.

Occlusion and the Sight-line

In ordcr for an ohjcct of intcrcst in a 3D information rcprcscntation to bccomc oc-

duded, ir =und objtxt mulit be pcwitioned mch that i& projecticm overlup thirt of

the first iu s e n h m the a 11pwi6c location, the vkwpoint. Ftrtbrmm t h ocdud- ing object must be located between the viewpoint anà the object of interest. These simple frictn pmvide us with m insight into h m we might seek ta develop a solution

by which we prevent the ocdusion of objects of interest. We wiU ciefine the siglit-line of a given object as the line segment cotwtxting the

centsr of that object tci the viewpaint. Tt in in the neighharbood d tbh sight-line that

other objects, which may occlude our object of interest w i l l Lie. AN new objecta of

interest are de6ned or the viewpoint is muveri ta a provide a new presenfrrtion of the infiormation layout the location of this sight-lîne within the layout changes, and the

set of objecb repmnting powible of occlmion wii l h g e tu well. The fa% that it ia anly in thh region, on or near the line of sight, that we will6nd

ohjectn repmnting patentid mluding ohj&~ i~ eignificmt; if thcire are na objmts in this neighborhd, other than the object of interest, tben we will bave no occlusion.

What WC am looking for thcn is a mcthod which will kccp tbc rcgion surmunding thc

si@-üne clelil of other occluding objectu.

Cuttinn planes, pasitioned and orient4 appropriately, could remm al1 of the data in ri mpmntation hetween the abject of inter& and the viewpaint. This woiild have

the desired &kt of keeping the region of the sight-line clear. However it would not support dctail-in-contcxt vicwing of 3D rcpmicntations.

Thmprirency tao could he u t d ta d u c e the effitct of amlitsion an our ~bility to

see the object of interest, at the expense of increased diculty in the comprehension of

the structure ae a whole, There are additional coets in rendering transparent objecte

correctly with a graphics APi such as OpenGLTM, as the use of the z-bder for visible

swfircr! determination iY no langer m û k h t . Trrinsparency may LJLSO be in- to

the point where the poteatially occluding ohjech are ernentidly remcwed h m the

sciene. T b would mompürrh the same d é c t t~ filtering. Navigation of the viewpoint to a new location will de6ne a new si&-line to the

objcct of intcrcst and change thc set of potentially occluding objccts. It may bc pot+

~ible to ûnd a n m extend viewpoint whm there rire h r or no ofcluding objectv

between the new viewpoint and the object of interest. In denser information repre sentationn (Le. wlumetric data) or representations where the distribution of elements

leadr to regions of higher and loww dewitieti (such ~iai wtter-plotu or graphs) it may

not be passible to find such a new viewpoht. Another mlution in this case is to py into the utnicture, moving the viewpaint past ptentially occluding ohjech. Thia hm the effect of shortenhg the sight-line and again reducing the potential set of occluders.

A side effect of this approacù is that least of the data in the representation will now

be outside of the viewing volume and thus culled h m the presentation.

What we seek to do is leave as much as possible of the original structure of the

representation intact. We develop a solution that constrains our actions to the neigh-

borhood of the sight-line and acts principally on those objecta which represent the

mwt likely potential octludm of an object of interest.

3.2.1 Towards a Solution

In order to construct our solution to reducing 3D ocduaion we w i l begin with the

2 0 + 2 0 translation function t(x) sccn in cquation 2.2. This function can hc applicd

to the redistribution of density mund a focai point in a 2D information repreyenta-

tion, as in (181.

if we extend the m u m of this fiinction from a point in a 2D mpmntatian to a

point in a 3D representation we can extend the operation of the translation function

from movcmcnt of clcmcnts in (2, y) to movlcmcnt in (2, y, z). This simple cxtcnsion is

capable of pmducing the local density reductions ohservecl in 1271 and h a Lsn wmt!

application to cluster-busting of 3D p p h or node layouts [59[ but yie.Ids little beneût

in more general visual representations wbere occlusion is a significant problem.

Table 3.1 illustrates the application of four well-known 2 0 detail-in-context Mew-

ing trmformtitiom and their extension to 3D through the addition of a x cornpunent

to the a lgor i th . The firat calumn of the taMe employs an orthogonal atretch n lp rtihm similar to that of t h Bifocal Diplay of Spence and Apperly [991. The second column illustratea the e f k t of a nonlinear orthogonal sttetching algorithm similar to

that found in Catgraph [55], Multi-Vicwpoint Pcnpcctivc [77] and thc hypcrhola of

Hyperbolic Speco [651. The third column hi distiiict h m the firrrC fan, due to the (ip

plicaiton of a radial application of the laput adjustment and magrdication funetion.

CHAPTER 3. METHQD 48

1 Stmtch Ortho. 1 Non-Linear Ortho. 1 Non-Linear Radial. 1 Step Ortho.

r** f - *- *:-*.?.. 6 , 2% .r* Lh - 4 0 m m

2D disp. 2 -.,.. ==- -\C.-W*CIC. . L. -&2. ..

Table 3.1: illustration of the application of four common 21) detd-in-mntext layout adjustment approaches to 3D layout via simple inclusion of 3rd dimension. In the k t row are examples of step and non-linear orthogonal stretching, non-linear radial displacement and non-space-filling orthogonal stretching. Row two illustrates the effcct of moving h m (2, y) to (2, y, z) for data and displaccmcnt function. Thc tbirri row shows the effect of the layout function without the accompanying magnification of nodes. Row four shows the displacement only &ect extendecl into three dimensions.

This function is similar to that employed in 3DPS [19] and Nonlinear Magnification Fiel& [57, 581. The ha1 column displays a step orthogonal algorithm similar to that

in the k m family of interfacm [4] and the more m n t Shrimp Views [101]. Excellent

surveys and examinations of the field of 2D detail-in-context viem can be found in

the worb of Noik [80] and Carpendale [23]. The firat row of the table shows the 2D magnification and translation functions

appiied in conjunction. In the second row these same functions are applied in 3D.

CHAPTER 3. METHQD 49

Note especidy that the 2D Iayoutu adjuritment d e m e s which minimize white-ypw..e

in the resulting layout maximim occlusion in the 3D c m . Row three removes the

magnification component from the algorithm and applies only the layout adjustment

component. This a p p r d simply transiatm the points in the graph in 2D without

djusting their individual d e . The bottom row demonstrates t b t the layout t n u g

formations which produce clear paths m m the data yield the c l e m t viRuai iicceg~

of the central, focal, point. This improved visual access is still ody available h m a limited set of viewpoints, and the object of interest wiU still be d u d e d h m many possible locations for the viewer.

3.2.2 Redefining the Focus

The principle pmhlem in auch d imt extensions of 2D detail-in-codext trasnforma-

tions to 3D is that they do Little to remive occlusion of the object of interest. As we

havc notcd, in ordcr to rcducc occlusion wc n d to rcmovc ohjccts from thc ncighbor-

hood of the tight-line. In the interest of maintainhg a detail-in-context pmientation

of the visual repmntation we seek to accornplish this without the removal of infor-

mation and with as little dimiption of the ownill ~tructure of the laput, m ÿi ih le .

This constrained adjustment ptesenres as far as possible the original mental mode1 of tbc 3D structure on thc part of thc user.

in Section 3.2.1 we tnated the object of internt itseielf as the murce b r the 3 0 extensions of our traditional 2D layout adjustment aigorithm. If, inatead, we deûne

the aight-iine Ecom the Mewpoint to tbe ohject of interest as the source of the trans-

formation function, then we can use a similar method to move objects away b m the

line of sight, mther than jwt away h m the object of intemt.

Figure 3.1 illustrates a 2D msmct ion of this mechruiim in operation. Fig-

tire 3.l(a) shows the original configuration of the information layout, the object of

interest (001) near the middie d the layout and the viewpoint (VP) at the Iower right. Thc sight-linc co~ccts thc 001 to thc VP. Figurc 3.l(b) shows thc displace

ment vec.t01~ genmted by the transformation function for the points lging on or near

the line of si&. Distance of each point is measured to the nearest point on the lise of

Figure 3.1: Operation of linear ORT in crosssection. Focal point and viewpoint dehe the line of Q h t thtaugh the structure (a). Diatmce of other elementa to line of ~igbt determines direction of displacement (b). The length of vectors in (b) will form the input into the function which detemines the magnitude of the multing displacernent vectors. Final transfonned layout pmducea clear line of sight from viewpint to focal point (c) .

sight. In determining these distances we also determine a d'iection vector, from the

uearest point on the line of sight ta the point l e h g adjusted. Points are nioveci in the

direction of these vectors. The length of the direction vectors forms the input to the

transformation function. The resdt of this Eunction is used to determine the degree of displacement for a point. Points closest to the Iine of sight are moved the furthest,

and points originally lying further away are moved in successively smaller increments.

Eventually a smooth transition is n d e to points wk& were far eriougli away as to

be undectd hy the tdormat ion . Figure 3.l(c) is the final configuration multing from the application of the transformation fuoction to the layout.

We will sutmequently d e r to operatoni such as this as Occlusion Reducing 'hns

formations of the visual representation, or OKï's. The dect of an ORT is to provide

a cieu line of sright, or v W wxw, to an object or mgion of i n ~ t within a 3D

visual representation by adjusting the layout. The application of multiple O m s may

be composed on a representation by combining the e&ts of the individual OMS on

the elements of the reptesentation.

We wiii refer to the @ t h e of cui ORT iui the munie of the function. ûther

definitions of the source are paaible as we will see in Section 3.3. The source of the

OilT ia the location which alementa of the mpmntation will mow away fmm ai the

ORT is applied. if we have a series of ORT operators O..n then a weighted average of

the effect of each ORTi can be employed where the iduence auother OW, (j # i) on

a point decrpm JM the distance of the point to the munie of ORTi dec~aws . Thin means tbat for points where this distance is O the duence of the other OEn;. is also O. Since the OOI h r ORTi defines one end of the aght-he for it will he at distance O

fiom the source of ORII;. We may also employ a simple average of the etléct of each O W i on an element, for d l i = O..n, as we do in the folluwing exampla.

Figure 3.2: Increlllling degree of appliclrtion of two OnIY to mal two abjects of interest (highlighted here with dsrLer color) in a 3D graph layout.

Figure 3.2 illustrates the progressive, simultaneous, application of two OKi's to a

3D graph in order to reveal two objecta of interest (blue), one neac the h n t of the

h q ~ u t , the second nearly crt the bock (as yeen from the current viewpoint). k a w

the viewpoint is an integral component in its mnst~ction, the 0#r remaim orient&

proprly and continues to pmvide occliinion reduction iw the viewpoint is m d .

Figure 3.3 illustrates the rotation of this repteaentation without the application of

O E s to rcvcal thc two focal points. Figun! 3.4 shows thc samc scqucncc of motion

with the O K b in place.

Figure 3.3: Rotation of the 3D p p h to illustrate the occilusion of t m objecta of internt (nocles highlightd with darlrer calor). A clerrr vkw of even the nemr of the two in the structure is availahle from only a limiteri range of viewpoints.

Figure 3.4: The sanie tliree viewpointa and same two ohjects of interest (now highlightd end increased in male for emphasis) with the application of OWs. Even the node at the fsr side of the graph is visible thmugh the ai@-line-clearing d e c t of the om.

In table 3.2 we retum to the examples of the sunple extension of the orthogonal

atretch, nonlinear orthogonal and nonlinear radiai aigorithm to 3D repmntationrr.

The f i row of table 3.2 shows the sarne 3 figures as the first three columnri of row

two in table 3.1. The hottom row in table 3.2 iHustnrtes the &ect of an OFK' on the

Orth. Non-Linear

Tahle 3.2: The effect of dding the effect of an OR!ï to the 3D extensions of some common 2D layout adjustment schemes for deteil-in-context viewing. In the first row of images we see the simple extension of the approaches to 3D: the central focai point is even more occluded than before the tayout adjustment in most cases. The second row adch the operabion of an QEtT to clear the Line of sigbt riom the viewpoint to the focal point.

layout, providing visual wm to the previously occluded nodes of interest.

The sightline is the simplest primitive we can empioy to produce an effective ORT. Likewise the nearest-point measurement in Euclidean spam is the simplest distance me8suremend. In order to laEilitate the description of a wider range of ORT operators we wii i ~~IWtruct a k d ~wrdinritt! *km (CS) for ewh OKï. The mation of a

I d ORT mrdiaate ey~tem ~~ two vectoin and a point in three riirneriniIr~lfi h m which we cm derive the porrition and orientation of the ORT CS relative to the mrld CS.

U'c will cal1 thc location of thc objcct of intcrcst asaaciatcd wit h an ORT thc focal point (FP). We wiil use this end of the Yight-line cur the location of the of the

Om CS. The direction from the focal point to the vkwpoint wii i form one of the

hro vectors needed in order to orient the QRT CS. For the moud vector we will u ~ e

the UP vector in the vie-, or camera, coordinate system. midy this direction

iri pmitive y, < O, 1,Q > or "upn in the world CS. In oder for the ORT CS to be

properly defineci we must ensure that the vectors VP-FP and UP are not parallel.

Figure 3.5: Annotated hramework of diagrams illustrating the relative shape of a selection of ORT functions. On the left (a) the OEtT Coordinate System (CS) z& is digned with the World CS 2-ais, on the right (h) the camera position (VRP) h a heen r n d and the OKT CS is ce-orieuted to track the change.

With these elementa we can construct a coordinate system that is cented on

the focd point, with the positive a axia oriented towads the viewpoint. By this

construction the x = O plane of the OKï CS contains the UP vector from the world CS. A rotation of the OKï CS amund the ~ght-üne i~ a simple matter of mtating the

C'P vector m u d the W-FP -or in the world CS and using this rotated vector in

the constmction of the OlYT CS. Figure 3.5 illustrates the codguration of the 0Ri' CS, World CS and viewpoint, or camera CS.

3.3 Distortion Space

In order to determine the effect of sn OR!ï' on a layout we trsnsform e d point p,

(2, y, z), into the OKI' CS via an affine transformation. This yields a new coordinate

p', (x', y', 2') in the OrYT CS. If the value of z' is greater than O then the point is

somewhere between the object of interest and the viewpoint. In this case the distance

iai m e d in the zy plane of the ORT CS only, which meammi the dbtantr! of the

paint to the sight-line. If the value of z' is leta than zero then the point b further

away h m the viewpint than the abject of intemt.

The advantage of the ORT CS is that the description of more complex distributions

of O W s is grcatly simplificd. Any transformation that will pmducc a mluction of

the dement demity dong the positive z tuch of the ORIT CS wiil rrchieve the desiml d t of occlusion reduction.

Thun frrr we have m n one dintrihution of &placements that we ciln cbiuacterize

as having a lineac-source and being tnuicated. This ORT operates relative to eight-

linc, thc z axis in thc ORT CS, and its distribution is truncatcd on thc far sidc of the

object of i n t m t h m the viewpint. ThiR producien the a cylindricd region of effeet

where the far end of the cylinder h m the viewpoint blends into a hemispherical cap.

In addition to such a linear-source function we may also desccihe an OKï that is

derived relative to either the y = O or x = O plane of the ORT CS. Each of these

plenes contai118 the z axis of the C)nr and therefore displacements of points away from t h e p lam wiii d u c e occlusion almg the sight-line M well as rrcmrn the plane.

It is ahm posRible to apply a transformation relative to di of the carciinal axes ar

planes of the OFKï CS in the same manner as they may have k e n C ~ P S ~ N C ~ ~ relative

ta thc cardinai axes of thc World CS (Figun! 3.6). If dcfincd rclativc to tbc OR!ï CS the deformationu will remciin ciligneci to the VRP.

k c a t i n g the distribution at the t = O plane is only one possible distribution of dinplacement throiigh the depth of the OM' CS. We may a h continue with the

constant application of the OKT dong the z-axki of the OIYT CS , or linearly scde

thc application of thc ORT so that it falls from it's maximum at thc ncar sidc of

the information layout to zero at the origin of the 0#r CS, or at the back of the

C W T E R 3. METHQD

Figure 3.6: Schematic of orthogonal stretch ORT. Distance of points is measured to oearest of the 3 planes passing through the focal point.

Figure 3.7: Linear extrusion through z a x i ~ of the functions desdbing the operation of a detail-in-context layout Nustment scheme. The Gaussian curve f (2) = e-l0-O2

forms t h hasis.

information layout. In figure 3.7 we see the Gaussian basis functioii extmded in a, in ôpm 3.8 the d e m of the function is scded to aero at the far end of the space.

If the O W is defined as having a plane source, then elements of the representation

will be pushed away h m this plane by the action of the OW. In this case the

distribution of the OitT ïunction acnws the plane, perpendicular to the z-exis of the

OKï CS, may also be modi6ed by a shaping function. This hinction controls the degree of application of the ORT in order to spatially condrain the transformation

CHAPTER 3. METHOD

O

Figurc 3.8: Thc samc graphs now illustrating thc cffcct of lincarly scaüng thc appli-

(b) Addition d Shaping I%iction

Figure 3.9: A secondary shaping function applied to the horizontal plane-relative OKï. Scaling in z is constant but the addition of the sbaping c m can be used to constrain the extent of the plane-relative function in x.

and thereby presem more of the original layout. These shaping bctions may be any

curvc that modulatcs thc dcgrcc of thc OKi' h m a wcight of O, no cffcct, to 1, thc originul eftect without shuping functioa. Figure 3.9 illubtrata the effect of a GuuÉElian

shaping function on an OKï defined relative to the y = O plane. The extent in width of the shaping fiinction may he a4jnsted independent of the degree of the OIM'.

Figure 3.10: Distance meariiirement according to the Lp metric in thwdimenirions.

As wit h the use of alternative distance metrics to achieve different distributions of

layout adjust ment in 2 D + 20 transformations, nieaaurement of distance accordhg

to diffmnt metria in 3D may be applied with a nimilar eflecit. For imtcuicc, we may

elect to measure distance with an Lp, rather tban Euclidean distance metric. The

conversion of a Mimensionel point to measurement with the Lp met& is shown in equation 3.1. If the profile of the ORT is computed with an Lp distance metric where

p = 1, then the ORT will have a dimond-shaped ratber than round appearance.

Inweming the value of the parameter p well beyonci 2 will shape the opening in a

progressively more squareci-off manner (Figure 3.10).

The use of a Super-Quadric distance metnc for modeling with implicit surfaces

was explorecl hy Tigges et al. in [102]. The conversion of Euclidean distance to Super-

Quadric distance is shown in equation 3.2, where the w and ns parameters- control the

Yhape of the qw't:. In determining &lrncr! kom s WWF, vwying these pliririmeters

pmvides independent contnil of the front-tt-hack and crm+wxtional pro6lct of the

shape of the basic ORT fiuiction (Figure 3.3.

To summarize, in the description of an OFYT operator we may d&e the basis function for thc transformation, thc sourcc of thc function, thc pmfilc of thc ORT

Table 3.3: The SuperQuadric distance metric allows separate speciîication of the w and eu shaping prumeten ta achieve a wider range of pmible met* spaces.

1 Parameter II Possible Values 1 1 hasis 1 linear, gaussian, hemisphere, hemisphere, cosine, 1

source

sbaping curve II constant, gaussian, linear, etc. varying (O..l) divtance metric Il Euclidean. LD (VI. Su~er-Quadric (ns.ewl

uscr-dcfincd O 5 j (z ) L 1 linear, plantu (horizontal, vertical or mtated hy u degrees),

z distribution

- - - -

Table 3.4: Parameters available in the definition of OH' operators.

cardinal axes, principle plmes constantt tnmcated, linear, short l inm

function dong the z axis of the ORT CS, the application of a shaping c m for

plane-relative OFtTs and the distance metric. We tabulate this parameter space in

table 3.4.

Table 3.6 illustrates some of the rang of such ORT descriptions via simplified

Linear Linear (new VRP1 Planar Planar (new VRP)

Talle 3.5: Some of the syace of OXT speciiications possible by varying the source a d distribution of the operator. The left column illuutrami OR!ïs defind relative to the z-axis of the ORT CS, the right calumn illustrata ORTs defined relative to the y = O and x = O plane of the ORT CS.

schematic diagrams. in each figure the ORT CS, the world CS and the viewpoint (camera CS) are indicated by triples of m. Two orientations of the viewpoint and 0R!ï CS are displayed in the world CS hr each combination of lunction source and distribution. The z axes of the OKï and w d d CS are paraiid on the left of each

image und an oblique viewpoint is shown on the right. The left colmm of the tuble illustrates OilT operators relative to a ünear source and the right column iiiustrates OltTs defined relative to the y = Q or z = 0 plane of the ORT CS.

Chapter 4

Applications

Having established a framework for the description of occlusion reducing transfor-

mations (OR%) of information layouts in 3D, we exmine the particular details of

applying these operators to a number of representations. We fint retum to our defini-

tion of three broaà categories of 3D information representations: discrete, contiguous

and continuous (Figure 4.1).

(a) Discrete (b) Contiguous (c) Cootinuous

Figure 4.1: Three classes of three-dimensional data representations

Discrete information tayouts include node and edge structures or 3D scatter-plot

layouts. These may be 3D graph layouts, or rnolecular models in the "bal1 and stick"

format. Rcprcscntations of this clam werc characterizcd as being spatially ordcced

CHAPTER 4. APPLICATIONS 63

where adjacency in the stmcture is illustrated by connections, such as edges, rather

than physical adjacency of the components.

The second category we defineci, contiguous information representations, included

3D models, ônite element sets, CAD data and so on. In these representations not

only was spatial ordering important but so was the physical properties of adjacency

and containment. ïkansformations of these representation involves consideration of

these properties, as the translation of components through each other dcfeats thc

presentation of the objects as 3D physical models.

The last class we defined included representations of datasets that were essentially

continuous in nature. That is the data rnay have beeen truly continuous, as the prod-

uct of 3D parametnc equations producing a volumetric function, or may have been

such 6nely discretiued datasets as to appear continuous, such as voluruetric meùical

imaging, geophysical or fluid-dynamics data. These datasets were generally rendered

with methods belonging to the field of volume rendering and present a specific chal-

lenge in dealing with their large sizes.

Throughout thc balonce of this chapter we examine the application of occlusion

rducing transformations to represeutatior*i belonging to e u h of these tliree b r d

categories.

4.1 Discrete Data Representations

Our k t class of 3D representations, discrete, is in some situations the class least

susceptible to the effects of occlusion in a 3D layout. In relatively sparse represen-

tations the likelihood of data elernents being arranged in such a manner as to result

in occlusion from any particular viewpoint is relatively low. There are a number of

situations though in which the likelihood of occlusion becomes an issue. Increasing

the number of discrete data elements in a particular layout increases the likelihood

that information elements will be laid out in such a manner as to cause an occlusion

situation frorn any particular viewpoint.

Local density variations causes clustering in regions even of smaüer discrcte Lay-

outs. This phenornenon and the use of local scale adjustment to improve the situation

CHGPTER 4. APPLICATIONS 64

is presented in [59]. In this system multidimensional data sets are represented as 3D

scatter plots agoinst axes represcnting a three-dimensionai frame of the nimensional

data. A straightfomd extension of Nonlinear rnagniiication fields from 2D to 3D

is applied in order to increase the apparent size of local clusters of data and thus

enhance the visibility of the spatial characteristics of the data.

4.1.1 Regular 3D Graphs Structures

In applying OWs to discrete information layouts we first use the example of a g3

element 3D grid-graph, as seen in figure 4.2. The regular spatial structure of this

graph lends itself to illustrating the effect of layout adjustment, as well as providing

a relatively dense, if uniform, distribution.

Figure 4.2: The original layout of the 9 x 9 x 9 3D grid-graph

The 3D lattice graph in this application has simple connectivity relationships

between nodes and theu nearest neighbors in x, y and z. The edges of the graph

are rendered to represent these relationships. We apply the turutable metaphor to

interaction of the viewer in this system and the viewpoint will normally be found

outside the bounds of the graph. For a structure of g3 nodes, a node in the center is


Likely to be occluded by nodes in 4 otber layers regardless of the choice of viewpoint.

(a) Data-aligned (b) Viewer-aligned

Figure 4.3: The orthogonal stcctch slg0ntb.m aligncd to the principle planes of the data layout spacc (a) and aligned to the viewer as an O W operator (b).

In these examples we color the nodes of the graph with a scale that ranges from

light grey to blue. The degree of the change to blue being proportional to the displace-

ment a node has undergone from its onginal location. This colonng has the effect of

illustrating some of the shape and distribution of the ORT operator through the cep

resentation. As discussed in Section 3.3 a wide range of combinations of ORT function

sources and distributions are possible. In figure 4.3(a) we see an orthogonal-stretch

layout adjustment algorithm applied to the g3 graph as we saw in table 3.1. Again the

central node is the object of interest but here the remaining nodes of the graph are

colored to illustrate the displacement t hey have expecienced. In figure 4.3(b) the same

fiinction is applied as an OKï operator, and now remains aligneci to the viewpoint.

In figure 4.4 we present an ORT that is definecl relative to the sight-line co~ec t ing

the object of interest to the viewpoint. In this instance the object of interest (001) is

the central node in the g3 graph and the distribution of the ORT has been truncated

at the position of the 001. The shape of this function is similar to that illustrated

schematically in the topleft image of table 3.5.

If the sight-line is extended through the node of interest then the ORT results in

a clear si&-line which isolates the node against the background, we see this result in

CHAPTER 4. APPLICATlONS

Figure 4.4: The 3D grid-graph witb the central node specified as the object of interest. An ORT has b e n applied to reduce occlusion. Color of the remaining nodes in the graph represent the degree to which they have b e n displaced by the ORT. The darkest nodes have been movcd the ma t .

figure 4.5(a). If the visual clutter of nodes behind the object of interest had interfered

with its examination t hen this pattern of layout adjustment distribution may be useful.

The shape of this functiou is similiu to that of the left image of row two in table 3.5.

Other possibilities include a tapered cone-like distribution of the ORT function

which is seen in figure 4.5(b). The shape of this operator is illustrated in the image

on the left side of row four in table 3.5 will be of more interest in the application areas

wiil discuss later in this chapter.

Choosing a plane containing the sight-line as the source of the displacement func-

tion provides a means of interactively cutting-into the stmcture and having this %ut"

follow the sight-line as the viewpoint is moved around the structure. The simple&

two cases of this form of ORT are vertically and horizontally positioned planes which

produce vertical or horizontal cuts into the representation respectively (Figure 4.6(a)).

Here the truncated or tiipered distributions tue particuiarly effective, creating a book-

like opening in the representation (Figure 4.6(b)). This method provides good visi-

bility in the spatial neighborhood of the object of interest, more so within the plane

CHAPTER 4. APPLICATIONS

(a) Constant (b) Linear i r i n g

Figure 4.5: Examples of constant and linear scaling of the application of the ORT dong the z axis of the OnT coordinate system. The constant scaling isolates the object of interest against an empty background while the linear scaling looks very similar to the line segment relative application.

than perpendicular to it. These images provide examples of the shapes of OlYTs seen

in the images on the right side of rows two and three of table 3.5 respectively.

4.1.2 General3D Node and Edge Structures

Rather than generate a set of more randomly manged 3D graphs, we will use a ready-

made set of examples Ecom chemistry. Bal1 and stick models of molecular structures

are a cornmon means of representing the chemicai compounds, an example of such

a structure is the cdeine molecule in figure 4.7. In many respects these structures

are similar to 3D graphs, excepth that here the length of edges tends to be shorter

and the number of &es incident on a node is limited by the bonding properties of

the atom. That said, these models are used to represent complex structures where

the geometry of the layout is potentially more pertinent to the interpretation of the


U

(a) Constant

Figure 4.6: ORT functions applied relative to a horizontal plane through the object of interest. Objects within the plane remain in plane while those above and below are displaced. In (a) the operator is data-axis relative, and does not track changes in the viewpoint. The operator in (b) is viewpoint aligned.

representation than in abstract layouts of 3D graphs.

As a relatively simple initial example we will deal with a rnodel of the chemical

composition of caffcinc (Figure 4.8). This molccule consists of only 24 atoms and

25 chemicd bonds, so occlusion is not a particular problem here. This allows us to

discuss the effects of the application of ORTs to this domain of representations. We

will then see the application of an ORT to a substantially more cornplex chemical

compound.

In these examples we represent the atoms as colored spheres. For example here

we see carbon atoms represented as dark grey spheres, hydrogen as white, oxygen

as red and nitrogen as blue. We select as our object of interest one of the oxygen

atoms and apply a sight-line relative 0RT function. We truncate the ORT at the

depth of the 001 so as not to disturb the layout of atoms on the far side. Now as the


Figure 4.7: Caffeine Molecule: Ca HiaN40z

Figure 4.8: Lviovcrnent of the vicwpoint around the cdeinc molcculc without the application of any ORT functions.

viewpoint rnoves around the structure the other atoms are gently deflected away from

the sight-linc and retum to their original positions as the sight-line passes by. This

effect is illustrated by cornparhg the sequcnces of images in figure 4.8 with those in

figure 4.9. In figure 4.8 the atom of interest is highlighted. Without the application

of an ORT this atom is occluded as the viewpoint is rotated about the structure. In

figure 4.9 with the application of an OR2 the atom of interest remains visible.

There is a choice to be made hem; whether or not to distort the edges representing

the the bonds between the atoms. The relevant trade-offs are between the i n c r e d


Figure 4.9: The oxygen atom indicated in (a) is selected as the atom of interest for a linear-source ORT. The same mwement of the viewpoint is performed around the caffeine molecule and this atom remains visible as other atoms are deflected away from sight-line.

cost of rendering edges as a piece-wise linear approximations of the curveâ paths which

they take through the ORT influenced space, and the detrimental effect straight edges

may have when they are not subject to the effect of the ORï. The immunity of the

cdgm from the ORT detracts from the effect of the OKï on the reprcsentation ns

a whole. in the current evample leaving the bonds undistorted means that even if

two atoms are displaced away From the sight-line in opposite directions, the bond

connecting them may remain in place or be rnoved into place, in front of the object of

interest. This may introduce a mal1 amount of occlusion, but it may create a great

deal of visual clutter in front of the object of interest.

As a more complex example we use the molecuiar structure of the vitamin B12

(C63H88C~N14014P). Here we select at random a particular oxygen atom as the object

of interest. In figure 4.10 we apply an ORT, increasing the degree of application over

severai frarnes. We use the same locaily constrained sight-line relative distort ion

function as in the previous example. The Detailed views in figure 4.11 provide a

clearer picture of the effect of this distortion on the local layout.

Other possibilities within this domain include the selection of chernical substnic-

tures as objects of interest rather than individual atoms. For example a benzene ring


Figure 4.10: Sequence illustrating the application of a linear-source ORT to the structure of vitarnin B12. The Oxygen atom selected as an atom of interest is in the region indicated by the overlay box.

Figure 4.11: A detail view of the region indicated by the overlay box in the previous figure. The result of the successive application of a linear-source ORT to the (initially hidden) Oxygen atom is iilustrated.

may form a structure of interest that would be cleared of occluding elements and re-

main undistorted as it's local neighborhood and relationship to the overall structure

is studid.

More complex representations of moiecular stmctures and particdarly proteins

are common in the biochemistry. Protein structures form comple.u spatial folding


arrangements, the intncacies of which are of particular interest in the function pr*

teins during biological processes. The convoluted structures of are often represented

visually as cibbons that illustrate the winding, twisting and folding of the rnolecular

chab which comprises a protein. These representations are often dense and involve

considerable occlusion issues. An interesting future area of work would be to apply

ORTs to the interactive investigation of these more complex visual representations.

4.1.3 Hierarchical 3D Graph Structures

(a)

Figure 4.12: A selected leaf-node in a cone tree layout of a directory stnicture is indicated by the overlay in (a). This node is brought to the front through concentcic rotations of the cone tree structure; (b) through (d)

Moving away from the biosciences and back to the realm of the information sciences

we can explore the application of ORTs to one more fom of 3D grnph layout, cone

trees, These structures provide a means of creating a 3D layout of a hiecarchical

information set. In a typical implementation of cone trees, specifying a node of interest

within the structure leads to the structure being adjusted nutomatically such that the

node of interest is rototed to the front-and-center position, as seen in figure 4.12. This

works well in the case of a single object of interest but the mechanism does not readily

extend to provide a means of dealing with two, arbitrarily specified, objects of interest.

If an additional step is taken in the interaction scenario then we can use OKïs

to support interaction with multiple nodes of interest with the cone tree h e w o r k .

CHAPTER 4. APPLICATZONS 73

One are for the application of cone trees is the display of directory and file structures.

If, in this case, a user is searching for a particular version of a file within the layout

a scan of the file system may yield several potential candidates. With single focus

operation each of the files produceù as a result of the search must be exarnined in

a sequential manner. With the addition of multiple ORTs, each providing occlusion

reduction for one of the search results, a multi-focal3D detail-and-context oveniew

is possible. This display facilitates the addition of more detailed information (file

date, path, author ...) to each result (either simultaneously if there are relatively few

resuits or as the user hovers the cursor if there are too many results for simultaneous

display).

Figure 4.13: Two leaf-nodes, labelled a and b in (a) are selected sirnultanmusly. Application of two ORT operators irnproves the visibility of these nodes without explicitly rotating one or the other to the front; (b) and (c).

Once the multiple objects of interest are defined navigation of the viewpoint is

possible whilc the objccts rernain visible, as illustratcd in figure 4.14. This is an

inherent property of each OR' incorporating the curent viewpoint. We see t his h m

a secondary viewpoint in figure 4.15.

Figure 4.14: Once ORT operators are attacheci to nodes a and b, in (a), these nodes remain visible during movement of the viewpoint; (b) and (c).

Figure 4.15: The area of influence and viewpoint alignment of the ORT operators in the previous sequence, as seen from a secondary viewpoint. The OFtT operators remain aligned to the primary viewpoint as it is moved around the cone tree.

4.1.4 3D Desktop Environment

Our final example for the application of ORïs in discrete information spaces is a

3D-desktopstyle environment, seen in figure 4.16. To demonstrate this application

we have implemented a prototype of such an environment on a Personal Cornputer

running the Microsoft Widows operating system. As the prototype initializes it

"grabs" images of each applications currently ninning on the users' desktop. These

Figure 4.16: 3D Desktop environment

images are attached to polygonal surfaces within the 3Daesktop environment in which

the user is able to navigate by movement of the viewpoint with a turntable metaphor.

Within this environment the user can arrange the 3D windows by dragging them, as

illustrateci in figure 4.17. A single window may aiso be brought to the bcal position,

immcdiatcly in front of the viewcr, where it nppears at the sarne scalc as it would on

the users desktop.

Users cannot currently interact with the applications in this environment; tbat

would repuire the construction of an operating system level redimtion mechanism as

was describeci in the Task Gallery system [go]. Three is also no means of interacting

with the operating system; either to launch new applications or terminate those al-

ready running. In any case the development of this prototype was an effort to explore

the application of ORT operaton within such an environment, rather than the creation of a fulIy-functional 3D-deshop system. Interactions within the environment

are restricted to the arrangcmcnt of windows in three dimensions and navigation of

the viewpoint.

After a user selects a window, either by clicking on it once or: when no window is

c m n t l y selected, by hovering the mouse aver it. that window becornes rnarkeà as an

Figure 4.17: As the selected window is pushed back through a cluster of windows in the 3D desktop environment the cluster is disperseci in order to prevent occlusion of the selected window.

object of interest. Once a window is mwked as an 0 0 1 an ORT function is applicd to

resolve any potential occlusion situations. In figure 4.17 the selected window is being

pushed to the back of the scene. This results in the sight-line moving through the

cluster of un-selected windows and the effect of the ORT is to move these windows

away from the neighborhood of the sight-line. Figure 4.18 shows the second and third

images of the previous sequence, as well as annotations to indicate the effect of the

O R moving windows away from their original locations. New ORTs are introduced

over a number of frames, producing a smoot h transition between the previous state of

the layout and the new layout. If a selection results in the t r a d e r from one object of

interest to a second then the original ORT is removed in a similar manncr, producing

a cross-fade between the two states of the layout. Were the layout to "jump" between

states the task of tracking changes in the layout would detract from the principle task

of interaction with the desktop environment.

Once a window has been selected as a focus by clicking on it, it remains the object

of interest until it is de-selected, either by clicking on another window of clicking

over "empty" space which de-selects al1 windows. As long as a window is selected it

rernains un-occluded as the user navigates through the space or changes the position of

CHAPTER 4. APPLICATIONS 7 7

Figure 4.18: Annotated images from the previous squence iHustrating the initial position (boxes) and movement (arrows) of the selected (solid line) and other (broken line) windows.

the window. As the user drags the selected window behind another group of windows

they are temporarily "pushed" off of the sight-line by the influence of the OKï as

seen in the sequence of images in figure 4.19.

The effect of using OMS on windows in such a 3D environment bears a strong

resemblance to the use of "Page Avoidance Behavior" in the Data Mountain (881 and

Task Gaiiery systems 1901 which we describeci in Section 2.5.3.

The actual distribution of layout adjustment in Data Mountain is determineci

differently from that with OELTs, witb each element of the layout in Data Mountain

seeking to maintain a minimum separation distance from al1 other elements in order to

avoid situations of occlusion. However the effect of maving a page through a cluster

of other elements which avoid it is similar to the effect produced by attaching an

O W to an element and repeating the scenario. We endeavor to convey the effect

of this action in figure 4.20. Here the setected page is moved ftom point a to point


Figure 4.19: As the selected window is moved from its initial position in the upper left of the view the cluster of other windows which it passes in front of lue dispersed by the action of the ORT attached to the selected window.

b dong the indicated vector in figure 4.20(a). The neacby windows are deflected

away from their initial positions as the selected window passes by, returning to their

initial positions after the selected window has passed. The clusters of arrows indicate

the sequence of deflection vectors generated over tirne as the selected window moves

through the sccne. The darker arrows are carlier in the sequencc, the Iighter arrows

later. Figure 4.20(b) illustrates the deflected positions of windows midway through

the movemcnt of the selected window.

4.2 Contiguous Data Representations

Our second classification of information representations in three-dimensions we termed

contiguous data representations. We characterized these representations as having

stronger adjacency and containment relationships than the discrete data representa-

tions we have just examined. Examples of contiguous data representations are 3D

modeb or p i assemblies in Computer Aided Drafting. Other examples would in-

clude surface data derived from wlurnetric data sets, such as medical irnaging data,

fluid dpamics, atmospheric or geophysical data.


Figure 4.20: Annotation of two frames From the previous sequence. As the selected window moves fmrn position a to position b the remaining windows are deflected by the action of the ORT. The m o w clusters in (a) indicate the progression of deflection vectors for the remaining windows. Early to late vectors in the resulting motion are shaded hom dark to lighter grey. On the rigbt (b) illustrates the state of the layout at the midpoint of the sequence. Initial ( p y boxcs) and final ( b l d boxcs) positions of the windows arc indicatcd as wcll as thcir multing displacemcnts (arrows)

In such dotasetu the layout is compriseci of mnponents which wili have physi-

cal relationships that may inciude containment or adjacency. ln the application of

ORTs to these tepresentations it may be necesaary to take t h e relationships into

account. This may mean anirnating components of a parts assembly through a partial

disasscmbly scquence beforc the parts corne under the inîiuence of the displacement

of the ORT. While requiring a somewhat more complex mode1 description, including

some information about the assemblage of parts, including containment relatiouships,

which parts must be removed before others are free to move and so on, the appli-

cation of OWs provides a means of creating an interactive assembly diagram. In

CHAPTER 4. APPLIC.4TIONS 80

such a systern other elements of the mode1 would disassemble themselves in order to

provide clear visual access to a component of interest. As the viewpoint is moved

component groups would disassemble and move out of the way then ressemble EU

necessary to provide occlusion-free views of the component of interest. increasing and

decreasing the magnitude of the Oms would also have the &ect of the assembly mm-

ing apart and reassembling itself. Previous work on a related system was presented

in [78]. In this systcm a rnock-up of a tisheye viewing system for assembly diagcams

was describeci. While including a level of detail function there was no support for a change in viewpoint. While we have not yet implemented such a system it remains

a ptomising application area for future work. To date we have only investigated the

application of OEtTs to dealing with simpler 3D models, lacking complex containment

and interconuection relationsbips between coniponents.

4.2.1 3D Models

We have implemented two systems which apply ORTs to component-based data. The

first system is geared towards 3D modcls consistiag of differcnt parts and WC use a rnodel of the skeletal structure of the human foot by woy of example. The second

system applies ORTs to surface data derived via the Marching-Cubes algorithm (see

appendix B) and demonstrates the use of ORTs to cut through components, or byers,

of this data to reveal underlying elements. These objects of interest are excepted frorn

the effect of the O W displaccments.

In Our fint system we apply ORTs to the examination of bones in the skeletal model

of a human hot, as seen in figure 4.21. This model does not bave any containment

relationships and the system as implemented is not sufficientty sophisticated to deal pmperly with models that would require disassembly sequences to deal with such

relationships.

By returning to our analysifi of detail-in-context views mated via 3D perspective

distortion, we can take the rnagnification producing aspects of transformations to p m

duce scaling of the components in 3D models. We will refer to these transformations

as Magnification f mducing Transformations (MPTs) and explore t hem in more detail


Figure 4.21: The skeletal model of the foot used in the following example. This model contains 26 separate components and 4204 triartgular faces.

shortly.

Applying a combination of ORTs and MPTs we can achieve results similar to

the transformations presenteù in the Zoom nlustrator [83]. Zoom illustrator was de-

signed and implemented as a 3D interactive medical illustration system. This system

incorporated a data-relative detail-in-context rnagnification capability derived from

the continuous zoom algorithm 132, 41 as a means of emphasizing particular compo-

nents. For instance if the first metatarsal bone is the current focus of attention then

other bones around it are xded and displaced in order to provide sufiicient space

to increase the scale of the bone of interest. These techniques are desccibed by the

authors as being similar to those applied in traditional 2D medical illustration.

By adding viewpoint-aligned O W s to the model we c m select a particular com-

poncnt of interest and reveal it via the action of the OW. We see this illustrated in

the sequence of images in figure 4.22. As the viewer navigates around the model, al1

of the remaining cornponents are dynamically deflected off of the sight-line. In this

manner a clear vies of the selected component is maintained as seen in figure fig-

ure 4.23. Figure 4.24 shows the model h m the same squence of viewpoints without


Figure 4.22: The extenial cuneiform bone (circled in (a) and highlighted in ail images) is selected as the focus and an ORT opcrator is i d to displace the rernaining 25 bones away €mm the sight-line.

Figure 4.23: Again the extemal cuneiform is the object of interest and remains visible in this scque11ce as the viewpoint rnoves around the model.

the addition of ORTs. Attention emphasis through scaling may be applied in conjunction with the occlu-

sion reduction of ORT operators. ln a mannec similar to Zoom iîlustrator we increase

the s a l e of the component of interest and displace, rather than d e , the remain-

ing components in ocder to provide d c i e n t room for the increase in scale of the


Figure 4.24: Again the external cuneifonn is the object of interest and remains visible in this sequence as the viewpoint moves around the model.

component of interest. WC illustrate this combination of actions in figure 4.25.

There are two potential mecbanisms to achieve this scaling. We may elect to scale

components in place, simply adjusting the local scaling factor as each component is

rendered at its original position. Alternatively we may elect to employ the effect of

perspective distortion to achieve scaling, using what we have previousiy termed a mag-

nification producing transformation (MW). In this technique components are moved

towards the viewpoint dong the sight-line through their geometric center in order to

ma& the component. We can also move components away frorn the viewpoint in

order to cornpress or minifSi them. We illustrate the operation of an MPT operator on

the modcl of the foot as scen from a secondary viewpoint in figure 4.26. Hcre WC sec

the model and a representation of the perspective viewing mistum. In figure 4.26(1>)

the focal component and those nearest it are moved towards the principle viewpoint

producing rnagnification. The degree of magnification depends on the ratio of original

and final positions relative to the z-axis of the camera coordinate system as outlined

in figure 4.27.

While substantially diierent in mechanian from in place scaling, the MPT method

prodiices similar resul ts once perspective projection has been applied. Figures 4.28(a)

and 4.28(b) show the skeletal model of the foot in two orientations with no ORTs


(a) N*Scaling (b) Scaiing for Emphasis

Figure 4.25: In (a) no scaling is applied, the effect of the O W is simply to displace components and reduce occlusion. In (b) we have subsequently scaled components according to their geometric distance bom the object of interest, the external cuneiform bone.

applied. In figures 4.30(a) and 4.30(a) both an ORT and a MPT are aùded to

provide occlusion reduction and perspective based scaling of the navicular bone. The

images in figures 4.31(a) and 4.31(bj demonstrate the same degree of displacement

and scaling, but here the scaling is produced by in place component scaling. From the

principle viewpoint in the perspective projection system there is no apparent motion

of the components in an MPT as they are constrained to move back and forth along

the vector of their original sight-line.

The m o d significant difference between the resulting images, produced by in-place

or MPT scaling are in cases where adjacent magnifiecl components begin to intersect

each other as seen in the detaiied view, figure 4.32(a). With a MPT, components are

separated in depth such intersections are resolved by the magnified components being

rendered in front of the compressed or less magnified elements. Partial occlusion of

the smaller elements in the overlapping areas is the result as we see in figure 4.32(b).

C W T E R 4. APPLICATIONS

(a) Perspective Viewing h s t u m (b) Mapificatian via Displacement

Figure 4.26: Figure (a) illustrates the basic configuration of the perspective viewing volume and 3D model. Spheres indicate the location of the viewpoint, the view reference point and the point rnidway between. Components of the model are translated dong their individual lines of sight in (b) to produce magnification via perspective projection.

The application of hlPTs in conjunction with depth-enhancing stereo vision s u p

port, whether via multiple screens in a head mounted display or more sirnpIy by

rendering the scene as a red-blue 3D maglyph leads to interesting perceptual cf-

fects. The more magnified objects now appear not only larger but cioeer than the less

magnified components. The efficacy of this approach for producing magni6cation in

conjunction with stereo viewing as well as an investigation of the perceptual effects

incurred provides an array of topics for future study.

Figure 4.27: The effect of decreasing the distance d from the viewpoint on projected scde in perspective projection. Final scale varies as the inverse of the change in distance.

Figure 4.28: Side (a) and front (b) vicws of the foot mode1 with the navicular bone seltxted as an object of interest and highlighteâ. No distortion or magaification bas been applid and the bone cemains al1 but completely occluded in these two views.

4.2.2 Isosurface Set Data

-4 second part of our class of contiguous representations includes isomirfaces derived

from wlumetric data. This information is often generated by an algorithm such as


Figure 4.29: Side (a) and front (b) views of the foot nidel with the riavicular bone selected as an object of interest and highlighted. Distortion only hrrs beeo applied to the layout of the rnodel, with no scaling for emphasis.

Yarching Cubes 1701. In mnny ciws thme surface extraction algorithms are applicd

successively to a data set in order to extract surfaces corresponding to the boundaries

of various different components. They may dso be used to extract surfaces from

several, spatially coincident, sets of data.

In medicd imaging for example, several passes may be made to derive separate

sets of surface data for bone, muscle, brain and tumor in a diagnostic cronial MRI scan. Figures 4.33 and 4.34 illustrate a selection of images from a diagnostic MRI scan and an aswciated lesion-mask, as well as the corresponding isosurfaces of skin, brain and lesion derived via application of the Marchiog Cubes [70] algorithm.

In dealing with this concentnc Iayer occlusion there is no way to disassemble the

components in order to provide clcar visual access to the interior features. The most

common appmaches to providing access to the interior elements of su& structures

are through the use of transparency, component removal or cutting planes. As we

have discussed earlier each of these approaches has some undesirable effects, either in

mmplicating the perception of the distinct surfaces or in that they remove considerable

quantities of information h m the display. Applying a rnodified, discontinuous, OElT


Figure 4.30: The navicular bone is selected as an object of interest and an ORT is applied to reduce occlusion. Simultaneously a small degree of masnification has been applied to emphasize the navicular bone and its ncighborhood. Magnification herc is produccd through perspective transformation and as a rcsult the navicular is rendercd in front of othcr boues that may have still resulted in partial occlusion.

to the layers which occlude a component layer of interest in such a display rnakes it possible to produce a viewpoint depeudent clear visual patti to the region of interest.

A discontinuous ORï operates on the representation at level below that of discrete

components or layers, acting on the individual polygons (triangles) that comprise these

surfaces. ïkangles are transformeci into the local coordinate system of the ORT and

the resulting, displaccd, locations of its vertices are determined. Discontinuous OiCs

are so far limited to planerelative functions. Triangles wbich span the source plane

of the function may be split into components entirely on one side or tbe other and

re-triangiilated, leading to a clean cut surface. Other less cornputationally complex

solutions inciude moving the triangle to the side already containing the majority of

vertices, or leaving the plane-spanning triangles out of the final image aitogether.

For example, a linearly-tapered vertical-planerelative ORT is applied to a repre-

sentation deriwd h m a diagnostic MRI scan of a Multiple Sclerosis patient and a

volumetric rnap of lesions. Figure 4.35 illustrates the composition of the 3 layew from

figure 4.34 rendered parti& transparency in order ta make the intemal layers visible.


Figure 4.31: The same two views O€ the human foot with the navicular bone as an object of interest in the layout. Hem maguification is produced via in-place scaling of the individual componcnts. The most apparent different is that in (b) the interior cunciform bone now partiaily occludcs the navicular.

Figure 4.36 illustrates the sequential application of an ORT to reveal the lesion layer,

pushing bnck the outer brain and skin layers and providing an occlusion-frce view of

a portion of the lesion mask. This deformation will riutometically follow the viewer

as the viewpoint is manipulated to examine the data from a different angle. In these

images the simplest approach to dealing with plane-spanning triangles was taken and

they are not rendered in the final image.

4.3 Continuous Data Representations

Often a volumetric data set will be amorphous and la& clear surface boundaries to

extract via methods such as marching cubes. Io these cases direct volume rendering

(DVR) algorithms are the preferred approach. There are a wide range of DVR meth-

ods. These algorithms fall into three major categories based on the method in which

they traverse the object to be rendered; image order, object order, or a hybrid of the

two.

CHtlPTER 4. APPLICATIONS

(a) in-Place Scaling (b) Perspective Scaling

Figure 4.32: A detail view of the area just in front of the navicular bone with in-place scaling (a) and perspective scaling (b). The intersection of the externai cuneiform and thc third metatarsal in (a) is resolved in (b) by the relative displacement of the cornponents in depth.

Image order DVR methods include re-projection, and ray tracing of the volume. In ce-projection voxel value are averaged along paraifel rays from each pixel in the view-

ing plane. The resul ting image resernbles an X-cay. Source-attenuation reprojection

assips a source strength and attenuation coefficient to each voxel and allowed for

obscuring of more distant voxels [96]. Reprojection is a simple case of ray casting

while applying a SLiM operator.

Ray casting of the volume involves performing an image order traversal of the

pixels in the images plane. Rays are cast from the viewpoint through each pixel and

through the volume (Figure 2.1), the opacities and shaded intensities encountered are

summed to determine the finai opacity and color of the pixel. Rays continue to traverse

the volume until the opacity encountered by the ray sums to uni& or the ray exits

the volume. When a ray intersects a cell between gxid points an interpolation may be

performed to 6nd the value at the intersection point (Figure 2.2). Ray casting, while

CPU intensive, produces high quality images of the entire data set, not just surfaces

as in surface fitting aigorithm such as marching cubes.


(a) Proton-Density Layer (b) T2 Layer (c) Lesion Mask

Figure 4.33: Example source images for the generation of Mmhing Cubes decived surfaces of MRI data.

-

(a) Proton-Density Layer (b) T2 hyer

1

(c) Lesion Mask

Figure 4.34: Three separate surfaces h m diagnostic MRi data of Multiple-Sclerosis (MS) lesions. Proton-Density layers (a) reveal outer surfaces such as the skin, T2 layers (b) reveal neural tissue (brain and eyes), while the lesion mask (c) indicates location of MS lesions. These three data sets are used in the demonstration of the application of an ORT to volumetric data visualization.

Ray casting of volumes was first developed as a visible surface algorithm buy

Tuy and Tuy [106]. The first multi-valued volumes were ray-traced by Blinn in [IO]

as a means of rendering participatory media. This technique was later extended

CHAPTER 4. APPL?CATIOlVS

Figure 4.35: Composite 4.35 is rendered as slightly transparent in order to make spatial organization apparent.

Figure 4.36: Sequence illustrating the application of an ORT to isosurface data. The lesion mask iayer (green) is not affected by the xaled and truncated planar deformation and is revealed as the outer layers are cut and pushed back.

and applied to scientific information and generd 3 D textures by Kajiya, Kay and

Von Herzen [54, 561. Lewy in [Ml presented a uethod that computed the partial

occupancy of a voxel by diierent materials and deriveci the color and opacity from

the various contributions.


Object order DVR methods are characterized as processing the scene in order of

the elements of the data set, rather than pixel by pixel (Figure 2.3). The cuberille

rendering Jgonthm [47] is a straightforward mapping of vaxels to six-sided polyhedra

(cubes). Hidden surfaces are normally removed when rendering with a z-buffer dg*

rithm [24] but it is also possible to determine a traversal order that yields the correct

visible surfaces since a volume is so strongly spatially sorted. These orderings such

as front-teback (421, back-to-front [37, 421 and octrcc based approaches 1751 al1 yield

a performance benefit. The blocky appearance of cuberille rendered images cm be

improved by shading the cubes according to gradient information rather tban their

geometry.

Splatting [117, 118) is an object order approach that operates by building up an

image of the volume in the projection plane. The process is often likened to building

up a picture by dropping appropriately colored snowballs representing the projection

of each voxel. When the snowball hits the plane it splats and spreads the contribution

of the voxel over an area.

We will examine the application of ORTs to two methods of object order volume

rendering which utilize 2 and 3D texture mapping, as well as the blending clipabilities

in OpenGLTiM to approximate the process of DVR. The first method we will examine

is fast-splatting (291.

4.3.1 Fast-Splat Rendering

Fast Splatting is an object order approach in which each voxel of data is rendered in

place as a small quad (4-sided polygon) which is colored by the volume data. A normal

derived from the gradient of the volume across that voxel may also be associateci with

the quad and used in hardware based illumination calculations. The quad used to

represent the vmel is further modified by using a dpha-texture map that performs

the function of the blending kernel in traditional splatting.

The resulting colored and alpha-mapped quad is rendered into the OpenGL frame-

buffer in much the same way as a traditional splat contributes to the final image. The

correct performance of this algorithm depends on the volume being traversed fiom

Figure 4.37: UNC head data set rendered via fast-splatting

back to front . Simply determining the axis of the data most parailel to the view-

direction and rendering planes perpendicular to that back to front means that the

planes and the rendered quadrangles which comprise them are now within 45degrees

of perpendicular to the viewpoint. Figure 4.37 illustrates a data set of a human head,

wi th the top and back of the cranium removcd to revcal the ou tcr surface of the brain,

rendered with the fast-splatting approach.

tn order to apply an ORT to the data the location of the voxels are transformeci

into the space of the 0 W CS. A translation vector is determined and applied to

vary the final, rendered, position of the individual quadrangles. Effects similar to the

discontinuous OWs d i s c d in Section 4.2.2 are achieved with the application of

linearly attenuated planerelative displacement functiona (similar to the l m r right

images in table 3.5). Such an application is illustrateci on the CT data of a human

skull in figure 4.38. These functions have the effect of pmducing a "cut-into and

retractn incision into the interior of the volume. The cxtcnt of thc incision can be

lirnited or rnodified by a shaping function to achieve a more constrained effect as

discussed earlier in Section 3.3.

The reasan for the use of plane relative incisions hem rather than simply employing


Figure 4.38: The application of a vertical-plane-source ORT to CT data of a human skull rendered via fast splatting. Observe the increase in brigbtness at the edge of the ORT-indiiced split. This is the result of splat primitives overlapping.

sight-line as we often did with more discrete forms of data, is that the "real-world"

meaning of a point incision, stretched out large enough to provide any interna1 visual

access is difficult to establish. The interior, or bounding, surface of a line-relative

incision would be formed by the intcrscction of a single ray with thc volumc data

and not reveal rnuch meminfil visuai infonnation. Conversely, the application of a

plane-relative displacement function produces incisions which have interior surfaces

produced by the intersection of the source plane and the volume data. These cut

surfaces carry much more useful infonnation and provide a virtual approximation of

a red-world incision.

Examples of the application of such ORTs to volumetric data rendered with the

fast-splatting algorithm are seen in the following figures. Figure 4.39(a) is an image

of an ORT applied to the UNC head data set. In the next image (Figure 4.39(b)) the

representation is rotateci without updating the viewpoint of the ORT in order to hi&-

light the shape of the interaction of the O W with the representation. Figures 4.40(a)

and 4.40(b) are again view aligned and o f k t images of another ORT, in tbis case

the extent of the function acrm the plane is truncated as well as scaling it linearly in


(a) View-Alignerl

- I I

(b) Secondary Viewpoint

Figure 4-39: .4 horizontal-plane ORT applied to the UNC Head data set. In (a) the OFlT is alignecl to the viewpoint. In (b) we have moved the viewpoint independent of the O W (disabled automatic tracking of the viewpoint) in order to illustrate the Iinear scaling of the application of the ORT in view-aligned depth. The ORT is scaled in depth kom the front of the representation to the depth of the region of interest.

depth to produce the wedge effect. In figure 4.41 we apply an OKï to the full-color

Visible Humm femde data set.

There are some visual effects that the simple transformation of the splat-producing

quadranglcs produccs ris they are rendcrcd. As quadrarigles are "pushed aside" to

make an incision into the representlrtion they have a tendeucy to pile-up and overlap

more than they did in the original data layout. The effect of this increasing overlap

is that in these regions there are additional contributions to the compositing process

achieved with the OpenGL blending mechanism. This results in increased intensity

of the color of the volume data in thcse regions. Figurc 4.38(a) illustrates a CT data

set of a skull rendered with the fast-splatting algorithm. Figures 4.38(b) and 4.38(c)

demonstrate the application of a constrained, plane-relative OEtT and rnakes apparent

the resulting brightening of the surfaces at the edge of the cut. Anisotropic scaling of

the quadrangles in the regions of compression around the OR?' could be applied to

reduce or eliminate this effect.


(a) View-Aügncd

-

(b) Secondary Viewpoint

Figure 4.40: The same data set and orientation of views. Here a sbaping curve has b e n added to control the extent of the ORT operator across the horiaontal plane.

4.3.2 3D TextureBased Rendering

The advent of high speed texture mapping hardware hes made the application of

tbis technology practicai for use as a method of direct volume rendering. Previous

approacties used some hardware-assisteci Gouraud-shading methods [97, ûû] by calcu-

lating projections of volume regions and then treating them as polygons in (coherent

projection).

The possibüity of using the rendering hardware of the Silicon Graphics hc. Reality

Eagine is raisecl by Akely in [Il. Subsequently a number of papers were prcsented in

rapid succession which al1 a p p r d e d this problem in sirnilar manuers [31, 43, 131.

Cullip and Neuman outline two approaches, described as object-space and image-

space [NI, Guan and Lipes examine the issues concerning hardware [43], and Cabral,

Cam and Foran describe the use of texture mapping hardware to accelerate Radon

transformations [13].

Wilson, Van Gelder and Wilhelms [120] examine the application of graphics li-

b r l (OpenGLT") routines to automate the procm of performing texture space


(a) initial (b) View-Aligneci (c) Secondary Viewpoint

Figure 4.41: The Visible Human Female data set with a plane-relative OR! applied. Here the ORT scaled in depth from the front to back of the data set, rather than from the front to the region of interest.

Figure 4.42: Relation of siice domain to volume data domain.

transformations and setting clipping planes. They employ a bounding region of tex-

ture sarnpling planes of a size sufficient to accommodate the texture data volume in

any orientation as in figure 4.42. Wilson et al. then use the Texture Transformation

Matrix and the 6 hardware clipping planes of the Reality Engine to render only the

parts of these planes that are witbin the volume for a given orientation. The use of

the hardware clipping planes has the advantage of reducing the size of the planes that

are rendered. It is the pixel-fil1 rate that greatly slows dowu the process of rendering


in this situation.

Data is initidy convertcd into a 3D texture map with a one-the application of

a transfer function to determine the red green and blue values as well as the opacity.

Color and opacity are stored in a 3D texture map. The texture map is applied to

many parallel polygonal planes, each plane sampling a slice through the texture, The

texture coordinates are specified for the corners of the planes and the texture mapping

hardware intcrpolates the texture coordinates across the plane in three-dimensions.

(a) Object Axis Aligned

Figurc 4.43: Two basic approachcs to the alignrncnt of siices in 3D-Texturc hardware accelerated volume rendering.

There are two means of relating the polygons to the 3D texture, the data volume.

Either the texture planes may be aligned with the principle axes of the data and

move (rotate/translate) with the data (Figure 4.43(a)), or the planes may remain

aligned parallel to the projection plane and the texture coordinates alone m d

(translated/rotate) in ordcr to view the data from a Oifferent position (Figure 4.43(b)).

In general more parallel planes sampling the 3D texture will result in a higher quality

image and l e s planes yields higher rendering speeds.

The advantage of this method is that once al1 of the data has been downloaded into


to texture memory and the polygons transformed the graphics hardware is capable

of performing al1 of the slicc rendering and the composition of the images. The

disadvantages are the restriction to rectilinear volumes and the relatively small texture

sizes that can be accommodateci in texture memory at one time. The process of

bricking (breaking the volume up into smaller texture regions and then loading them

into texture RAM and rendering them sequentially) permits the use of this technique

to render larger volurncs and also providcs a method of optimizing the rcndering

process, by constructing bricking layouts that eliminate regions of the original data

set that are cmpty, these regioos are then not rendered and no time is Iost in computing

the texture coordinates and cornpositing in rendered pixels where no volume elements

are present.

Silicon Graphics Inc. has also developeù a library of supporting routines to fa-

cilitate the application of this technique with the OpenGLTM graphics library'. The

OpenGLTM Volumizer API extends the OpenGL set of primitives to include points,

lines, triangles and now tetras (tetrahedrons). Five tetrahedrons form a minimal

tcssellation of a cube and tetrahcdrons are able to tcsscllatc any 3D shapc. Thus in

order to render any arbitMnly defined region of 3D volume data the volume rendering

pipeline need only be able to render the tetrahedral primitive.

Mechanism

In order to provide a means of producing ORTs which result in the apparent ciitting

into actions we saw in Section 4.3.1 we must provide a division of the tcxturc snmpling

surfaces which provides additional vertices in the regions where displacements will

occur. Our initial approach here was to find a method of fitting a mesh to a function

that described the shape of the displacement function.

The method for anisotropic mesh generation presented in [Il] provides a means of

producing a tessellation of the plane that 6ts the geometry of the triangulation to a

space defined by the Hessian of the function. The Hessian is the matrix of the second

'OpenGLTM version 1.2 dl contain 3D texture coordinate generation as a core part of the API, in 1.1 it is available only on machines supporting the 3D textures as an extension.


Figure 4.44: 2D Gaussian Function j(r) = e-10~o+2-io.oy'

Figure 4.45: Hessian of 2D Gaussian Function f (x) = e-'0.022-'0~0~3

partial differentials of a function. For instance the Hessian of the threedimensional

gaussian function f (x) = e-'O-Od-LO.O? ,figure 4.44, is presented in figure 4.45. Using

the met hods presented in [12] we were able to generate meshes which provide a region

of increased detail around a point of interest and conform ta the shape of the Hessian

of a gaussian function centered at that point, as in 6gure figure 4.46.

The method of rendering volumes using 3D texture mapping hardware will opti-

mally see the texture sampling planes rotated to remain perpendicular to the view

direction. We c m use this fact to our advantage in the generation of ORTs with this

method. If we create a single tesseliated mesh which provides the desired geometrical

Figure 4.46: Anisotropic mesh aligned to Hessian of Gaussian function.

(a) Object-Axis Aligneci (b) Sigbtliie Aligned

Figure 4.47: Sampling planes aligned to data space axk (a) or centered on sight-line (b)

detail for manipulation of the volume, rather than simply lining al1 of these planes

up, centering them on the view direction vector through the volume, we cau instead

position the planes so that t h y are centered on the sight-line througti our point of interest. We see these two alternatives in Ggure 4.47.


Another interesting optimization is possible here that was not possible in the

prcvious method of applying OWs to fast-splatted volumetric data. Li we dcterminc

tbe maximum deformation of one plane by the O W , from this we can derive the state

of ail of the remaining texture sampling planes. The state of any plane can be derived

by interpolating between the state of the initial plane and the maximally deformeci

plane to the appropriate state. In this manner we may produce functions that are

constant, tmncatcd or tapeced with varying dcpth in the ORT coordinatc system, al1

from the state of two texture sampling planes.

Figure 4.48: Configuration of tessellatcd plane and hidden texture surface used in demonstrating stretch approach to ORT application.

As with the fast splatting method of rendering volumes, here too the application

of a linear-source function is questionable. The first issue arises around introducing

a symmetric hole into the tessellated surface and the sutsequent remangement of

texture coordinates to accommodate the hole as it grows. Again the result wouid be

a tube, the inncr boundary of which wouid simply be the series of voxels that a given

ray intersecteci on its path through the volume. This would convey little meaningful

information. One possibility solution we have explored is what we term the colored

balloon approach to introducing a hole into the tessellated surface. In this case tri-

angles whose edges are stretched by the deformation have their contribution to the


Figure 4-49: Progressive application of deformation and resulting transparency effect. -4s triangles are stretched they are made progressively less opaque. The result is that in the area of the deformation the background layer becomes visible.

Figure 4.50: Detail view illustrating the transition of opacity values at the boundary of the deformation which results in the blurry appearance.

composition operation reduced by decreasing their alpha proportionaily. Thus trian-

gles that are stretched become increasingly transparent. To illustrate this mechanism

we set up a tessellated plane in front of a background image as seen in figure 4.48.

The result of decreasing the aipha of the sttetched triangles in the mesh is illustrated

in figure 4.49 with the edges in the mesh rendered and in figure 4.50 with the edges

removed. The most significant problems with this method are that it results in an

inner boundary to the distortion which is fuzzy and indistinct and that it results in

the removal of some information from the display. If information removal was deemed

acceptable then the trimming of the tessellation to produce a hole would result in


clearer imaging of the interior boundacy of the deformation and the result would be

something like a CSG operation removing a subvolume of data h m the final im-

age. This would still have the tidvantage that the removed regions would track the

viewpoint as the representation is manipulated and the regions of interest are moved within the volume.

Rather than deal with a linear-source deformation we will concentrate naw on

the application of planorelative deformations of the volume data set. As with fast-

splatting these deformations produce the appearance of an incision into and retraction

of material in the representation. No data is removed in this manner it is merely

pushed aside in order to produce interior visual access aligned to the current viewpoint.

Figure 4.51: The initial configuration of the slice sarnpling mesh. Iliangulation density is increased in the inside corner where OW displacements will occur. This min- Unizes the extent of linear interpolation of texture coordinates.

Moving from a linear-source to a plane-source we can modify the way in which we

arrange the texture sampling planes, replacing each single plane with four quarter-

planes. These four planes covering the four quarters of the original plane. We wiU

dso change the tesvellation pattern to provide increased geometrical detail in the

inner corner of the quarter-plane which will be adjacent to the line of sigbt through

the region of interest in this revised scheme. We see this configuration in figure 4.3.2.

Al1 of the benefits of reduced computation of deformations remain true for the

use of quarter planes. In fact computation of vertex deformations now need only be


Figure 4.52: Introduction of a semi-circular deformation of the texture wnpling mesh by deforming vertices along the y axis.

Figure 4.53: Mirroring the single deformed texture sarnple plane allows the creation of a closed empty region in the middle of the plane.

performed for the maximum deformation of one quarter plane, the remaining three

being rendered by reflection of the first quadrant as we seen in figure 4.53. Texture

sampling coordinates are dso determineci for only one plane and the remaining three

planes obtain the correct texture coordinates by manipulation of the OpenGLTM

texture coordinate transformation mat r i . . This use of geometrical transformations

allows for the generation of an entùe slice from a mesh cavering only one quadrant.

As described in Section 3.3 we have the capabilil of modi lng the profile of the

ORT in depth and in the plane perpendicdar to the view direction. This dlows us

to produce an ORT that is spatially constrained across the field of view and using

a hernispherical profile as a shaping envelope produce ORTs that resemble incision

CIWPTER 4. APPLICATIONS 107

and retraction operatioas on the volume data. We illustrate this in figures 4.52(a)

through 4.52(6). The inner boundary of the OFlT surface is formed by the intersection

of the ORT source plane with the volume data. The use of quartered texture sampling

planes means that there is a pre-defined break in the geornetry at the location of

this intersection and this obviates the complexity of dynamically re-triangulating the

texture sampling planes.

Figure 4.54: OpenGL clipping planes are used to trim the texture planes to the boundaries of the volume presentation space

Having devetoped a method for the construction of suitable geometrical sampling

surfaces it is necessary to integrate these polygonal primitives with the 3D texture

data in order to produce the votume rendered image. A point of interest may be

located an-where withii the bounds of the volume data, and the volume data rnay be

oriented arbitruily with respect to the viewer. Our method of ceritering the texture

sampling planes on the line of sight from the viewer and through the point of interest

means that the sampling planes must be scaled sufkiently large enough to encompass

the data volume in the most extreme combinations of point of interest position and

orientation. This means a point of interest in one corner of the data and an orientation

of the data with rotations of 45 degrees in two axes to the viewer, such that a vertex

of the data volume points towards the viewer. In this configuration the projection

of the data volume is maximized in width, height and depth. The dimension of a

single tarture sampüng plane mud then be the maximum diagonal across the data

set, and the stack of texture sampling planes must span that same distance in depth.

CHAPTER 4. APPLICATIONS 10s

In generai this means that a large portion of the area of each texture sampling plane

fdIs outside of the buundaries of the data volume. Rather than wasting computational

effort perfoming pixel-fiIl operations in these areas we apply clipping planes (rotateci

appropriately to account for the orientation of the viewer and data volume) to trim

the stack of sampling plana ta the bounds of the data volume in a manner similar to

that employed in [120]. Figure 4.54 illustrates the clipping of the tasellatecl texture

sarnpiing planes and thcir rotation perpendicular to the vicwer. In figure 4.54(d) we see the addition of the effect of an ORT function to the planes.

Figure 4.55: Progrssive application of ORT to produce a horizontal, shaped, opening in a singb plane in a volumetric representation.

Figure 4.56: Progressive application of ORT to produce a vertical, shaped, opening in a single plane in a volumetric representation.


Figure 4.57: Increasing the width of the shaping function to enlarge the horizontal ORT in a single slice of a volumetric data set.

Figure 4.58: Texture transformation matrix is manipulated so that as the intenec- tion of the sampling planes is moved across the presentation space the texture space remains stationary.

The texture coordinates for each of vertex in a given texture sampling plane are

computed b a d on the position and orientation of the plane within the data volume.

These coordinates are determined for the vertices in their original, un-deformeci, con-

figuration. These same coordinates are used in the application of texture to the de-

formai planes resulting from the application of nn ORT operator. The result is that

the data from the original position of the vertex is pulled to the deformeci position.

Rather than explicitly deforming each of the volume data elements as with fast splat-

ting we are able to achieve an interpolateci result between the vertices of each element

of the triangular mesh. Appropriate application of ORT operators and modification of


the texture transformation matrices allows for the creation of horizontal (Figure 4.55)

or verticai (Figure 4.56) boundcd (Figure 4.57) or unbounded planerelative incisions

into the volume data. Mhermore the movement of the point of interest is accom-

plished by the movement of the textureci plane and the munter translation of the

texture coordinates, maintainhg the volume data position(Figure 4.58).

Figure 4.59: The Visible Human Male data set rendered via 3D-texture slicing.

We will illustrate the effect of ORT functions on the head of the Visible Human

male data set, shown in its initial state in figure 4.59. Examples of the application

of a bounded, linearly-truncated, plane-relative ORT are shown in figures 4.60(a)

and 4.60(b). In figures 4.61(a) through 4.61(c) the head is rotated in place while a

honzontai ORT function provides visual access to an area behind and between the

eyes.

The next set of examples employ the UNC head data set; figures 4.62(a) through

4.62(c) illustrate the application of an ORT to the UNC head data set. In an oblique

presentation (Figure 4.62(a)) both a verticdy aligneci and horizontally aligned ORT are demonstrated. Arbitrary orientations between horizontal could be obtained by


Figure 4.60: The application of a horizontal ORT to the Visible Human Male data set. The point of interest is behind the left eye and the effect of the ORT is to reveal two cut-surfaces aligned to the viewpoint witbout the removal of data.

Figure 4.61: A more centrally located point of interest is specified in the Visible Human Male data set and the viewpoint is moved around the head fiom the front to the left side.

rotating the up vector used in the construction of the ORT coordinate system and

rotating the texture sarnpling planes around the sightiiie to accommodate the new

configuration.

CHAPTER 4. APPLXATIONS

Figure 4.62: The UNC Head CT data set with vertically and horizontally aligned ORT tùnctions applied to reved cut surfaces aligned to the current viewpoint.

Of course the method we have d e d b e d here applies only to a single region of

interest in the volume representation and corresponding ORT. Having a single ORT source means that we can arrange the texture sampling planes dong that source by

shearing their positions to center them ou the sight-line through the point of interest.

To extend the system and to provide support for multiple regions of interest and ORTs we must abandon some of the efficiencies we have employed. Since the location of

the intersection of multiple OWs with each successive texture sarnpling plane would

diverge as we moved away €tom the viewer, we can not employ a single tessellation of

these planes that provides additional geometrical detail in specifically the tight places

in d l planes. Rather a compromise solution of sufficient detail thmughout the plain

would be desirable. Each of these planes would have to by dynarnically intersected by

the ORT source planes and cut and ce-triangulated at the line of intersection. While a

gceat deai more computation is required at run time such a system remains plausible

as an area of future work. Interestingly the fast-splatting method requires no such

extension to account for multiple ORTs, since it is essentially a very dense example

of the same methods that are applied to render discrete models.

4.3.3 Temporally Sequential 2D Information

Another source of 3D information is the change in a 2D information layout through

time. A 3D layout of such information is possible when temporal sequence is employcd

as one of the spatial axes in a manner similar to that demonstrated in [3]. Figure 4.63 is

an example of such 2D-over-time data arranged to form a 3D cube. We have employed

this method in the Tardis system [21] for the display and exploration of spatie

temporal landscape data generated by the SELES (Spatiaily Explicit Landscape Event

Simulator) engine [Ml.

Figure 4.63: Arrangement of spatio-temporal data as a bdimensional cube by using a spatial mis to represent time.

One of the metaphors employed in Tardis for interaction with such 2D-over-time

information is that of a fiipbook. The data is presented as a cube, where 2 axes

represent space and the third time. By cracking the cube open perpendicular to

one of these axes two interior faces are reveaied, representing adjacent slices through

the data. We see the result of such an operation in figure 4.64. if the cube is split

perpendicular to the temporal axis then the faces display the state of two spatial

dimensions across a step in time at the position of the split. If the cube is split across

one of the spatial axes then the changes dong a line across the landscape through

tirne are revealed.

An operator derived fiom the ORT can be applied to the interaction of a user

with this display metaphor in order to maintain the visibility of the open pages of the


Figure 4.64: A block of spaticAempora1 landscape data and an OIYI: operator applied to reveal the state of tbe landscape at an instant in time.

book. The application of an ORT nieluis that each of the two faces will rernain visible

to from the Mewpoint during manipulation of the split position or navigation of the

viewpoint as we see in figure 4.65. Adjusting the position of the opening reveals a new

point in tirne or space, while the viewpoint may be repositioned in order to obtain a

clearer view of one face by orienting it perpendicular to the viewer.

Figure 4.65: Positioning a split in a data-cube (left), applying an 0FtT operator to reveal two internai faces (middle left), repositioning the viewpoint to obtain a more perpendicular view of the right face (middle right) and finally selecting a new point in at which to position the split.


The ORT operator takes into account the position and orientation of the data

cube, the opening in the cube, and the position of the viewpoint. We modify the

OM' so that the splitting plane across the cube foms the source of the OM' operator,

regardless of the relative position of the viewpoint. A line €rom the intersection of

this plane with the far side of the datacube to the viewer becomes a tool for the

determination of the degree to which to apply the ORT function. If the viewpoint

lies on the plane splitting the cube a relatively small degree of distortion reveals the

two inner faces of the split. If the viewpoint lies away tiom the splitting plane then

the degree of the ORT function is increased such that this sightline lies between the

two open face.

I

Figure 4.66: Operation of the book mode OFtT with the hardcover appearance.

Two modes of operation are possible in this book-like configuration of an OR'.

We identify these modes with their similarity to manner in which hard- and softcover

books behave. In operating as a saftcouer bwk the two sections of the cube formed

by the split are sheared away €rom the viewpoint and the near faces of these halva

may become compressed, the far face of tbe cube remaias plaoar. In operating IM a

hardcouer book the two sections of the cube are rotated about the intersection of the

splitting plane with the far side of the cube as seen in figure 4.66. In this case the two

sections are not sheared, their near faces do not cornpress and the far face is broken

into two acrm the bend.


in each mode the relative sizes of the two sections produced by the split provides

information about the relative position within the dataset in a manner that is familiar

to us h m our experiences with physical books. Animating the tuming of pages as

the position of the split is adjusted rnay also be employed to further support the book

metaphor. Browsing through such a structure by moving the splitting plane supports

tasks such as examining the structural changes in a landscape over time.

4.4 Discussion

This work presents a new framework with which to describe transformations on a data layout. The effect of these transformations on a layout is distinct from a change

of the layout itself. Supporting the perception of these transformations as such will

be an important aspect in their effective application.

As with 2D layout adjustment approaches, an understanding of the effect these

operators have on a structure can be supported in a nurnber of ways. If the structure

is initially very rcgular (for cxample the 93939 grid graph in section 4.1.1) then the

effect of the ORT on the layout is readily apparent, even in a single still image. If

the structure of the data is more random (for example one of the molecular models

in section 4.1.2) then the eflect of the adjustment performed by the ORT may not

be so readily apparent. In these situations the addition of a secondary, more regular,

structure to the presentation may aid in the perception of the distinct effect of the

ORT. In section 4.1.2 we àid not deflect the patb of the bonds in the molecular models.

Bending these otherwise straight edges under the influence of the O W also provides

some additional clues as to the role of the layout adjustment operator on the original

structure.

ORT operators support constraineû layout adjustrnents which leave substantial

parts of the original data layout intact. Further properties such as color, scale and

orientation of components remain invariant under the effect of an ORT. Other prop

erties of groups of components such as mplauarity rnay not be preserved, although

maintenance of orthogonal ordering is supporteci.


Comprehension of these distorted layouts may be supported by a number of dif-

ferent mechanisms and properties of the distortions themselves. As in 3DPS these

distortions support both the concepts of reversibility and revertability as describeci

by Piaget [al]. Revertability is the understanding that two states are related and that

one can effect manipulations to move Eram one to the other and back while reversibil-

ity is the idea that two states are in some way quivalent. The ability to move between

the original and distorted states of the the layout is an important aspect in supporthg

understanding through these mechanisms. The fact that the adjustment of the layout

is spatially constrained and that as the viewpoint rnoves different regions enter and

exit the area of this influence hirther supports the perception of revertability.

This ability to move the viewpoint or re-orient the layout leads to the generation

of motion fields through the movement of individual features of the structure. The

interaction of the ORT with the initial layout overlays a second set of motion vec-

tors. These additional motion cues sumund the area of interest but do not affect

the actual object of interest, at the source of the OW. This isolation of the focal

objcct in a secondary motion field may serve to further emphasize the location of the

object of interest. An important m a of future work will be to conduct studies of

the fundamental aspects of perception and cornprehension in interacting with these

operaton.

Chapter 5

Conclusion

The application of 3D mmputer graphies to information presentation is a field that

continues to evolve and diverge rapicüy. We have examineci the field of detail-in-

context àisplays for 2D information repmentations, and their extension to 3D infor-

mation spaces. We saw that tbese techniques do not deal directly with the problem of

ocvliuiion of objectsi of intemt which occurs in 3D representationu. We have ah ~ e e n

that pmviou approaches ta reducing mcluaion in 3D do not produce detail-in-context

mults. We have pretmted a layout Rdjiintment a p p m h ta creating 3D detail-in-

context views, derived from 2D oriented techniques, but accounting for the unique

challcngcs of 3D. Since the concepts in this m r k w m fini pmnted in [27] we have wen mlated

reaults in the work of a nurnber ofother mearchers; notabiy dimntinuous ray deflec- tom [62], and page amidance [MJ. Wile differing in t h i r undnrlying mwhanism, these techniques mk to produce similar results to thoee we have seea ia t h appli-

cation of our own ORTs to volumc data and 3D documcnt spaccs. What WC havc

accomplished heir! is ta mnstruct a h e w o r k within with we c m d d h e the oper-

ation of OKïs as well as related systems.

5.1 Contribution

Tbr! mmt pjgniiicant concept we hope to have brought famard iri the comideration

of the sight-line of an object of interest in creating 3D detail-in-context views. The

phenornenon of occlusion p m ~ n t s a challenge specific to 3D repmentations. In order

for detail-in-context tools to be truly dective in dealing with 3D repreaentations

occlwion of the obj- of inter- mu& be dealt with. We believe that out solution,

the maintenance of cl= sight-linm ta the abject of inhwit through operators which

are inhemntly viewer-aligned in their description i~ a mlrition that provides an novel

and elegant approach, which extends readily to application m088 a wide range of

application domains and rcprcscntation stylcs.

Future Work

This work repreeents a beginning. There remah significant challenges and opportu-

nitics for thc future. Somc of thc most significant challcngcs involvc thc crcation of

intuitive user interiam for ys~ytem employing ORTY in vianiirlizing and interacting

with 3D representations. If this can be accomplished it will facilitate the study of

the U.W of thme operatorci in 31) interaction, and bopefiilly point t o d s the use of

ORT-like mechanisms in many areas of 3D visualization.

Our carlicr work in thc crcation of dctaii-in-contcxt vicwing twls for 2D data prc-

sented signi6cant challenges in developing meaningful metaphorci for direct interaction

of users with such piiable s u r h . While moving a lens around an information space hy clicking-and-dragging is intuitive, affordances for specification and adjustment of

other parametas of therre lenses (degree of magnification, f d and contextual extent,

lem shape adjutment) cemain open problems.

The challenge^ of providing dordances for the pif icat ion and adjiuitmeat of

OM' operators through direct manipulation iri an equally chdenging problem. Pmgrerin

in this a n a wii i he necessary in order to move us to a point where we can begin an indcpth cxarnination of thc interactions of uscrs with opcrators such as thcsc. WC

takr! encouragement h m the apparent yuccetréi of related toolv such as the page iivoid-

ance aspect of Data Mountain, and hope that experience will be repeated in testing of OKîs with even mare complicated qmtems mich as vnlumetric repme~trrtiom and

3D models.

The use of ORT operators in volume visualizatiou applicatious, esyetially d i c a l imaging, will rquiie lurther study and dewlopment in mnjunction with the damain

users and experts. It cemains to be seen if users such as radiologista will accept OWs as an alternative to methods such as sequential &ce presentation and traditional

cutting plane operations.

The challenge of opplying OM' operotoru to 3D part^ amemblim remiiinrr an intriguing m a for mare dewlapment. The probhm of collision detection and the

incluclion af (dis)asnemhly Requence information in madel reprerientatioas ail appear

to be solvable. The end result of an interactive assemMy diagram presents an at-

tractive goal. A similar systcm for thc intcractivc exploration of cornplex protcin structures is equdly intriguing.

5.3 Final Thought

Cyhcrspacc, thc ahstract rcalm of information rcprcscntation within thc computcr, is

a 'lpace where rb&mtions and interactions with information are pornibk i~cluding

those that we could never experience in the %al world". Our exploration of the

space of possibilities results in many methods that are readily comprehensihle, familiar

mappings of real world operations. More abstract, creative, exploratory designs must

continue to point t o d techniques chat art! new and novel in onler that we wey

discover the full potential of this medium.

Appendix A

3D Perception

We indude here an examination of the phenornena contrihuting ta the perception

and understanding of a 3D information representation. The presentation of such a scene may be a single 2D presentation, a projection of the 3D scene, a stereo-pair of

such images, an interactive image or an interactive stereqmb. Single image or single

display stem Li ah pwible with o varie@ of technologies, ranging h m red-blue

lilten to isolate left and right-eye images, or the use of polarizing filterri to pedorm

the w e ta&. The use of LCD-display rnoiuitd lenticidrv lenm to efféct alite

stereoscopic perception is a more ment development in this field, requiring no special glacnscs on the part of thc vicwcr [107]. To datc, howcvcr, most uscrs will havc acccss

only to the mwt basic form of nYud interface with the cornputer through a single

CRI' or LCD display device. in this more specific domain there rernain a number

a l deptb-ciim that enmurage the perception of a single image as that of a 3I-l mene. These features of an image and their role in 3D perception are examined by Kelsey in [Gû]

A.l Perceptual Cues

Occlusion is the phenornenon in 3D information ptesentations that we are moet inter-

ested in M i n g with in this work. Occlusion, or interyosition, in a 3D scene is very

much a reai world phenammon. As such the c a r n t pmntation of vhible d a c e s

APPENDIX A. 30 PERCEPTION 122

is an important element in the culriilt~ction of a 3D pmntation of information in

computer graphies. A wide range of techniques for achieving the correct occlusion re- latiomhip between elementa of a 3D m u e e i s t in cornputes graphim. Ohject-space

techniques such as list-priority aigcsithms siich as depth-sort [79] binary-space par- titioning trees [38] and iuiage s- techniques such as Zbuffering [24] are couimon

in interactive graphim while vkible ~urEscc! niy-tracing i~ mare mmman for phab

realistic rendering. The correct occlusion of more distant objects by nearer one, and

the correct presentation of the visible madaces of ail objects are al1 vital to the per-

ception of a 3D scene as coherent. Occlusion is one of the most important depth cues

a d a b l e in understanding a y c m .

Other c u ~ that play a role in the perception of rr 3D m u e include phencimena

aich as: motion parallax and kinetic depth effect, ahadhg and hadowing, pempm-

tiw distortion and relative size, texture distortion, stem disparity, convergence and

accommodation. Thcsc dcptb cucs have han cbaractcrizcd as primary : having to

do with the phpicd pmceuli of luoking at the scwne (binocular dispcuity, convergence

and accommodation) and 8econdaFy: having to do with the cognitive and p m g -

nitive taks of interpreting an image (perspective, size, tmtiue? shirding, shdow,

motion) (601. These secondary deptb cues are dm teferreci to as pictorial depth cues

as thcy am thc only cucs prcscnt in pictorial imagcs of 3D sccncs.

Motion pnrdax ia the pheoommoo of neet objecb appearing to move more thm

more &distant o b j ~ t s in an image during movement of the observer in the world ( e p

œntnc motion). Kinetic depth effect refers to the phenornenon of otmervers recovering 3D structure of objects that are undergohg rotation when viewed aa only a 2D projec-

t i o ~ As un example if a shadow of an object i~ projected ontu a back-lit screen, and the ohject ia mtated, the o k m r will inbrpret the changing shape of the shadow rrn

due to the change in orientation of tbe object, rather than as a 2D change in the ahape

of the shadow. Even a hasic fllireframe presentation of an object, lacking sh&g or

pcrspcctivc cucs, as in figurc A.l will bc pcrccivcd as a thrclodimcnsional structure if

the user iY able to move either the object or the viewpoint.

Shading of objects refm to the debermination of varying illumination a m t a the

mirface of an ohject anci principaily d s information about the shape of inindidiid

F i Al: Wùefrsme images with no depth information

surfaceri. Shadowing m l t a from one ohject blochg light fmm a directional mum h m falIing on a second ohject. Cast shdowa can he naerl aa a highly effective eue

in rewaling the relative placement of uhjects within a scene. Figure A.2 incorporates

a numbr of pictonal depth cuear including peqmtive projection, ~hirded Mirfrrrm and cast shadows. Wbile surface shading is a feature of the most common graph-

im pmgramming languages, such rn C l p a G ~ ~ ~ , the phenornenon of shaMng is l e s well supported and more difiicult, and computationally expenaive, to implement.

Rcodcring of thc actual boundarics of thc shadow volumc of a componcnt, rathcr

than simply the r d t i n g illumination chan* on other surfaces, hm b e n applied

by Ritter [87] in a rnanner similar to Silk Cursom [122] aa an aid to detemininfi the spatial relationship of individual components.

I

Figurc A.2: Imagc containhg scvcral dcptb cucs.

APPEiVDiX A. 3D PERCEPTION 124

Perspective distortion, through the application of a perspective projection of u

scene, provides an image in which elements of the scene exhibit similar distortions,

perspective foreshortenhg and the convergence of parallel lines, as we experience in

otmewing the real world. Figure A.3 is an example of the type of image commonly

useci to illustrate the effect of perspective on the perceivexi size of objects iii a scene.

Although the cyliidem are different &a in the 2D image the plyrwlcf! of Rtmng

perspective cues in the rest of the scene (the convergiag paralle1 lines one the floor

and d l ) le& us to perceive tbem as b e i i the eame size and their displacement in

deptb accounting for the apparent Mefence in size.

Figure A.3: Perspettive Illusion

The synthetic camera mode1 commoniy employed in computer gaphics lacks the

optical saphistication of the human v i s 4 hardware (the eye). For that matter the

synthetic computer graphics camera lacks the sophistication of even a basic optical

camera. The synthetic camera is parameterized by its position and orientation in

a scene, and hy its geometric field of view and aspect ratia. The geometric field of

view is the visual angle of the scene subtended by the location of the viewpoint in

the scene, looking in the direction of the view reference point. What is most ladting

in the synthetic camera is a depth of Md, or focus. AU objects in a typid image

produced with 3D computer graphics will be in focus, regardles of depth in the

scene. In a real camera, or in the eye, lesses are adjusted to hcing objects of a certain

APPENDIX A. 3D PERCEPTION

&ance into fociui. Objects neam the bnu and further awiyr wii i appear blurry and

out of focus. Ptincipally this has to do with a perceived la& of realism in computer

graphia. Additionally, in the human vision system, the oiitnf focus components of

the images in the left and right eye contribute to the determination of the relative

&stance estimation of objects t h u g h accommodation.

Figure A.4: Stem Viewing

The efftlct of orientation and dirrtance h m an okrver on the surface texture af

ohjecta plays a role in the ability to determine the orientation and shape of surfaces

in 3D scenes. Oneutatioii and scaling due to varying depth pduces variations com-

monlg m h e â to tu texture gradients that aid in the interpretation of a acene and

APPENDIX A. 3D PERCEPTION

(a) Lbt Eye Vicv (b) Right Eye V i

Figure AS: The simulated view from the left and ri& eye, including depth of field aud perspective foresbortening dects.

acid to the iealism of an image. In figure A.6 the gridded square on the left is shown

mtated in the center image. The dect of the orientation on the texture has a strong

influence on the perceid orientation of the surface. In tbe image on the cight the

la& of gradient in the texture (the grid) has the effect of making the image appear

as a Bat trapezoid ratber than a rotateà square.

Figure A.6: Texture grsdient &ect

Stereo presentation of separate images to the left and right eye cantnbiitw to the

perception of depth through stereopsia image pairs are generated with an of& to

APPENDIX A. 30 PERCEPTION 127

mimic the inter-ocular distcin~~ cind the d t i n g dirrpority of eiemenb in the imageai

is interpreted as the mult of dinering depth of elements in a p~ocess d e d stereopsia Figure A.? is a stempair of imaga lacking any other depth cues. CVhen stere&iruan

of the image pair is achieved by crossing the eyes, such that the lefk eye is l o o b g

at the nght image and the righ eye the le& image, the region of dots higbliglited in

figure A.8 will appear to float in front of the badrground dah. Stereopi~ irr eflectivr!

only in the near field of view, within 10 meters or so. A related phenornenon in 3D

perception, convergence, arises when the eyes move as a pair to target an ohject. The two eyes center the object in their respective fields of view and the result is that

the orientution of the eyes f o m a trimgle with the object ut the vertex. Focushg on abjects at Merent dhtruices will change the angle of convergence, n e a r objecta

forming a greater angle. When the e y a h t e an object at a wry great dintance they

are esentially paraIlel.

Figure A.7: Stem Pair

Motion h a widely studied visual depth cue [113], and hacr k a claArifirici in i

number of dierent mannem. Wallach and O'Connel1 [Ill] demonstrateci the ability of ohsemm to recmr 3D form from rotating 2D objects and labelled it the Kinetic

ûepth E f i t , this effect is even present in the absence of any other depth mes, as

in the motion of rotating dot patterns [93). Principally motion parallex is divideci ioto motion that rermlts from the movement of abjects in the mue, pasJive motion,

APPENDIX -4. 3D PERCEPTION

Figure A.8: Floating region

and motion of the scene that resuita h m physical movement of the observer, active

motion. Another way to characterize motion is by whether it is generated manuaily, by the action of the observer, or automatically. Uanually produced motion may be

active, through head- tradcing and virtual-reaiity techniques or peesive, using a device

aucb as a mouse or 6 DOF controller to control the orientation of the view or objecta

in t h mne. Many atuditsi induding thm by Ware Nid Rank [112] and Huhona

et al. [49] have found that manual motion, whether active or passive, is much more dective in supporting a number of 3D tsde (path tracing or ohject cornparison).

Appendix B

Marching Cubes

In this procexi valumetric data i l treated as celh, diacrete memurements at the vertices of a regular 3D grid. Figure B.l illustrates the remit of applying the marchimg

cubes algorithm to an implicit surface m d of two point sources, the fields of whch merge to form a characteristic peanut shape. Figure B.2 illustrates 3 views of the

&ILCI! multing h m the application of the algorithm tu the L'NC h d &ta set,

and figures B.3(4 ruid B.3(b) illwtrate the marching cuba surfam d e r i d €rom the

.&in layeni of the visible hiunan male (left) and female (right) data zieh.

Figure 13.1: A simple equipotential surfsce through an implicit model.

129

APPENDTX B. MARCKING CUBES 130

(b) Axia (c) Off-.eaS

Figin! R.2: UiïC Head Cl' data set renderd a~ an hiirface.

Figure B.3: The United States National Library of Medicine Visible Human Project data sets. The male 1).3(a) and fernale D 4 h ) data seta are derived h m axid slices of the visible data. This data obteined from the NPAC/OLDA online data source.

The algorithm searches the edges connecting these vertices for instances of the value crossiiig a pmieteniiiiied threshold level, i.e. somewhere dong edge .AB connecting ceih A and B we know that the field repcesented hy the data points d

APPENDlX B. MARCHINC CUBES 131

the thmhold Iewl if A A= t iuid B i=t or vice-versa. This a p p d dependu

on the sampling density of the volmetric data, and WU be unable to reconstruct a

change in the field that crociaes the t h h o l d Ievel twice ( A md R are hoth

greater or l e s than the threshold ia this case). The point dong the edge at which the

crosaing occurred is deterniined by interpolating, usually linearly, the values between

A and B to find the point at whirh the mult equal~ t. Repeating t h appmsch for dl 12 of the edges of a cube connecting 8 of the adjacent cells in the data set leads to the

ahility to determine a linear approximation for the intersection of the 3D volumetnc

field at which the field value quais t with this cube. By labeling each of the 8 vertices

in the cube tis a bit, ib W pcwaiible to comtmct ir code for ecrch @ble combination

of vertices king imide or outciide of the boundrvy formed hy the thmhald t. Thici le& to 256 different pmible ca~es in the configuration of the surf' r d n g the

cube and hence 256 diirent possible meane of triangulating that craxing. Symrnetry

rclationships cm bc applicd to d u c c that data to 15 casa which

figure B.4.

arc illustratcd in

Figure B.4: The 15 basic cases of edge c m i n g in the Muchhg Cubes algorithm.

The name marching rvbes cornes h m the m m e r in which t h t n r v e d of the

volume is couducteù. %amhg a m sequential colmas, and then planes of the data, reu~ing the mults of the croRRiag determination h m the previaiin mlumn or plane. Susîace normal data for the reerilaing polygonal surfaces may either be derived

fmui the cross producta of the mnstmcted triangles or in& fmm the interpolatd

from the gradient of the volumetrie field at the m m h g points. Dividing c u h ia a

reiateà means of creating a geometric model of an i m d a c e thmugh a vo!umetric data aet. Ratber tban deriving a polygonal approximation eutface the diividng cuhes algoritbm begins with the o h t i o n than in many casea the the of the tendered triirnglw bgha tu approach the ~ ize of a pixel in the hl image. Dividing cuberr

operateri by subriividing the volume in the region of the imurface until a single point

pmvidin a good vi.sud repmmtstion of the ~iirfam ak tbrit pixel in the renderd image. Normal data is simultaneously interpolated h m the gradient of the volume

ta that sarnc point.

Bibliography

[l] Kurt Akeley. Reaiity engine graphies. In Lynn Valastym and Laura editors, Proceedings of the Annuai Confemœ on Computer Cmphics, pages 109 116, New York, NY, USA, Auguet 1993. ACM Preas.

[2] Keith Aiidrews. Visualisiig cylwrapace: information visualisation in the harmony internet m r . In S#mpoSiwn on Injownation Visudùdion, pages 97-104, Atlanta, Ckt 1995. IEEE.

[3] H. Harlyn Baker. Computation and manipulation of thdimensional surfam h m image sequenm. Viswludion in scientific computing, pagm 109 127, 1990.

[4] Lyn Bartram, Albert Ho, John Dill, and Rank Henigman. The continuous zoom: A constrrrined fi~heye technique for viewing and navignting Irugt! information spaces. In Proceedinp of the ACM S ~ p o a i u m on User Interface Software and Technologp, Information Navigation, pages 207-215, 1995.

[Fi] Renjamin R. Rederrwin, Lamy Sted, and Jmes D. Hollm. Pad++: Advances in multiscale interfaces. ln P r o c d n g s O/ ACM CHI'94 Conference on Human Factors in Compding Syatems, volume 2 of SHORT PAPERS: Virtud and Visual Envinmmenb, pages 315 316, 1994.

[6] Michael Benedikt, ed. Cyberspoce: First Steps. MIT Press, Cambridge, MA, USA, 1991.

[7] T h h e r s l e e , Robert Cailliau, Ari Luotonen, Henrik Frystyk Yielsen, and Arthur Secret. The World-Wide Web. Cammunicatiuna O/ Me ACM, 37(8):76 82, hugust 1994.

[8] f . Bertin. Smiology of ~mphics: diagroms, networh and nKcpY. hiveni ty of Wivcoorrin P m , 1983. ( h c h edition 1967, t m l a t e d by William J. Berg 1983).

[9] Eric A. Bier, Maureen C. Stone, Ken Fier, William Buxton, and Tony D. DeRose. Toolglass and magie lenaes: The a e e - t h & interface. In Proceeding oj Siggmph '99, Computer Graphicri Annual Canference %ries, pagm 73 80. ACM, Aupst lM9.

[lQ] J. F. Biinn. Light reflection functions for simulation ofclouds and dusty surfaces. Computer Gmphicq, 16(3):21 29, Jidy lW2.

[Il] FFank J. b n . hiriatmpic mah generation with pacticlea Technical Report CMU-CS9b134, CS Dept., Carnegie Mellon University, May 1996.

[12] Frank J. h n and Paul S. Heckbert. A pliant metbod for anisotropic mesb generation. In 5th Inü. Mahing Roundtable, pagea 63 74, October 1996.

[13] Brian Cabral, Nancy Cam, and Jirn Foran. Aderated volume rendering and tomographie reconstruction using texture mapping hardware. In Arie Kauf- man and Wolfgang Knieger, editors, 1994 Sympoaiwn on Volume Visualitath, pages 91 98. .4CM SIGGRAPH, October 1994.

[14] Brian Cabral and Leith Casey Leedom. lmaging vector fields using Iine integral convolution. Computer Crogliicq 27(Annual Conference Series):263 270,1W3.

[15] Stuart Card, Jock MacKiniay, and k n Shneiderman, editors. Reodings in In- formation VWoEion: Wang Vieion #O Th& Morgan Kauünann PuMicrs, 1998.

[16] Stuart K. Carri. Visuahhg retrieved information: A survey. IEEE Computer Gropka and Appliwtiorrs, 16(2):û3-û7, M a d i lm.

[17] Stuart K. Card, George G. Rolwrtson, aiid William York. The webbook and the m b forager: Video use ~ c ~ d w for a mrld-wide web infornation mrkspache. In Piriaeding~ oj ACM CM96 Corifertnce on Human Fadors in Cmputing Sgstems, volume 2 of VIDEOS: World Wide Web, pages 416 417, 1996.

[la] M. Sheeiagh T. Carpendale, David J . Cowperthwaite au d MargarebAnne D. Storey, and F. David Fiacchia Exploring distinct aspects of the distortion viming paradigm. Technical &port Tn 9748, School of Camputing Science, Simon Fraser Uni-vemity, Durnabg, BC, Canada, Septembet 1997.

[19] M. Sheelagh T. Carpendale, David J. Couperthaite, and F. David Fhtcchia 3dimensional pliable surfaces: For the effective preaentation of visual information. In Proceedtngs of the ACM Symposium on User Inte$ace Sopolarie and Technolugg, Information Navigation, pages 217-226,1995.

(201 M. Sheelagh T. Carpendde, David d. Cowperthwaibe, tind F. David krihia Multi-de viewing. In BRan Blau et al., editois, V i s d pmœdinp: the wt and interdiaciplinary program of SICCRAPH 96: SIGGRAPH 96, Augwt 4 9, 1996, New Orleo~, LA, Cornputer Grmphirn, p q p 149 149, New York, NY 10036, USA, 1996. ACM Press.

[21] M. Shelagh T. Carpendde, David J. Cowperthuraite, M. T i , A. Fall, , and F. David Flacchia. The tardis: A visual exploration enviconment for landscape dynamics. In SPIE, Conjerience on V i s d Data Eqlorcrhrcrhm and An&& VJ, 1999.

[22] M. T. Carpendale, Andrew Fall, David J. Cowperthwaite, Joeeph Fall, and F. David Racchia Visual access for landscape event hied temporal data In Roni Yagcl and Grcgory M. Niclmn, cditors, Prucedinga of the Conference on Visudùation, pages 425 428, Los Alamitos, October 27 November 1 1996. IEEE.

[23] Slieelagh Carpendde. A h n i e w r k fur Elwtic PreYentutiorr Space. PhD thesis, Simon Rwr Uniwruity, 1999.

[24] Edwin E. Catmuii. A Subdivision Algorithm for Cornpufer Displug of Cuwed Surfaces. Ph.D. Thesin, UniwmiQ of Utah, December 1974.

[25] Mtrtthttw Chirlmem, Rabwt Ingnun, md ChriYtoph Pfiiinger. Adding image- ability featurw to inTormation displaya. In M i n g s of the A CM Synposium on User Interface Software and Tachnology, Papem: Inform~tian ViRualization, pages 33 39, 1M.

[26] Mei C. Chuah, Steven F. Roth, Joe Mattis, and John Kolojejchick. SDM: %ltxtive dynamic manipulrrtion of vhdizations. In h c e d i n p of the ACM Symposium on User lnterjace Software and Technology, 3ï) User Interiaces, peges 61-70, 1995.

1271 David J. Cowperthwaite, M. Sheelagh, T. Carpendale, and F. David Fracchia Visuel access for 3D data. in Pmœedinga of ACM CHI 96 ConImce on Hu- man Factors in Computing Syatem, volume 2 of SHORT PAPERS: Alternative Methda of Interaction, pagcs 175476,1996.

[28] Kenneth Cox, Stephen G. Eick, and Taoaong He. 3d geographic network displays. Sigmd Record, 2 5 ( 4 ) : M , Dcccmbcr 1996.

[29] Rogcr A. Crawfis. Rcal-timc slicing of data spacc. In IEEE Viswliz<ition 'a. IEEE, oct 1996.

[30] Cmliiia Cm-Neira, Diiniel d. Sandin, und Thomils A. DeFiinti. Surround- screen projection-based virtual reality: The design and implementation of the CAVE. In Jamev T. Kajiya, editar, Compuier Cnaphics (SICCRAPH '93 Pm- d ings ) , volume 27, page^ 135 142, August 1M3.

[31] Timothy -1. Cullip and UIrich Neumann. h l e r a t i n g volume reconstruction with 3D texture hardware. Technical Report TR93-û27, Department of Com- puter Science, University of nerth Carolina - Chape1 Hill, may 1994.

[32] J . Dill, L. Birrtrrun, A. Hu, and F. Henigmiui. A continuoudy viuiabh m m for navigating large hietarchical networlrs. in Proceedinp of IEEE International Confemnce on Systems, Man and Cyberndics, 386 390, W b e r 1994.

[93] Kim M. Fairchild, Steven E. Poltrock, and George W. Fiiraas. SemNct: Tbree-dimenaional graphic represeatation of large knowledge bases. In Ray- monde Guindon, editor, Cogniiiue Science and its Applimtiona for Human- Computer Intemction, pages 201 233. Lawrence Erlbeum Associates, H W e , New Jeresy, U.S.A., May 1988.

(341 Andrew Fall and ,laieph Fall. b u t y iind the b e d : St!plurrting specificlrtion hom implementstion for mdels of landscape dynamics. Technical Report TR 199405, Simon Fraaer Univemity, School of Camputing Science, apr 1999.

[35] S. Feiner md C. k h e m . Worlds within worlds: metapham for explorhg n- dimensional virtuai mrl<Es. In ACM, eciitor, UIST. Thini Annual Synposium on User Interface Soflware and Technolugy. ProE8edinp of the ACM SIGCRA PH Symposium, Snowbird, Utah, USA, Octuber 3 5, 1990, pages 76 83, New York, NY 10036, USA, ûctober 1990. ACM Press.

[Ml Jarries D. Foley, Andria van Dam, Steven K. Feiuef, and Joli11 F. Huglies. Computer Craphics Principles and Proctice (2nd Ed.). Adhn-Waley, lm.

[37] Gideon Frieàer, Dan Gordon, and R ilnthony Reynolds. 13ack-tdront (BTF) display of wnrei-based objects. M E R Computer Cnaphica and Appiicutions, 5(1):52 60, January 1985.

[38] H. Fuchs, Z. M. Kedem, and B. F. Xaylor. On visible surface generation by a priori tree structures. In Computer Cmphies (SIGCRAPH '80 fmxeding~), volume 14, page^ 124 133, July 1981).

[39] George W. b a s . The FISWYJ3 view: A new look at stmctured files. Tech- nical Memorandum #81-11221-9, Bell Laborotories, Murray Hill, New Jemy 07974, U.S..4., 12 Ornober tM1.

[40] George W. Fumas. Generetized Weye views. In Marilyn M. Mantei end Peter Orbeton, editors, Pmceedinp of the ACM Contmnœ on Human Factors in Cornputer Systems, SIGCHI Builetin, pages 16 23. k i a t i o n for Computer Machinery, New York, U.S.A., 1984.

(411 Jonas Gomes, Lucia Dana, Bruno Costa, and Luiz Velho. Warping and Mor- phing of Gmphiml Objeck. Morgan Kaufmm, lm.

[42] Dan Gordon and R. Anthony ReynoldR. Image! qme shding of Sdimemiond objets. Computer Vision, Grnphics, and Image hmwing, 29(3):361-376, March 1985.

[43] S.Y. Guan and R. Lipes. innovative volume rendering using 3d texture map ping. In SPIE Mediaù I-ng 1994, d u m e 2164, pages 382-392. SPIE - The International Society for Optical Eagineering, 1994.

[44] J. Hamel, R. Michel, and T. Strothotte. Vibility thtough inaccuracy: G e metric distortion to reduce the cluttering in mute map. In Winter School O/

Computer Cmphics 1996, Fcbruary 1996.

[45] David Harel. On visuai hrmaiisms. Communications of the ACM, 31(5):514 530, May 1988.

[46] R J. Hcndlcy, K. S. Drcw, A. M. Wood, and R. Bcalc. Narcissus: Viualising information. In Nahum D. Gershon and Steve Eick, editors, P m . IEEE Symp. Information Viwolization, InfoVia, pages 9û 96. IEEE Computer Soc. Press, 30 31 Chtaber 1995.

G a b r T. Hem,an aaci Hsua K m Liu. The-dimensional display of humm OtgallB from computed tomograms. Cornputet Cnaphics and Image hocessing, 9(1):1 21, Jaauruy 1979.

[48] Ken Hinckley, Randy P a d , Demis P d t t , and N d F. Klrciciell. Twd~anded virtual mauipulation. ACM h ~ a C t j O ~ on Computcr-Hurnan Intcmction, a(3):26(t302,1998.

[49] G ~ ~ y S. Huhana, Gregory W. Shirah, ruid David G. Fout. The effectn of motion and stereopsis on three-dimenaional visuaülation. International Journal of Human-Compuler &dies, 47(5):60947,1997.

[50] Victoria Interrante, Henry Fuchs, and Stephen M. Pizer. Conveying the 3D Shape of Smoothly Curvîng %wpent Surf~~e8 via Texture. IEEE 3hinsoc- tioru on Visuoihtion and Cornputer Gmphics, 3(2):98-117, April 1997.

[51] Victoria Interrante and Chester G r d . Strategim for dwtively viriualizing 3D flow with volume LIC (color plate S. 568). In Roni Yagel and Hans Ha- gen, editoa, PIIoceedings of the 8th Annud IEEE Conference on Viuduation (VISU-Sr), pagm 421 424, Loci Alamiton, Octoher 19 24 1W7. IEEE Computer Society Press.

[Fi21 Victoria Inhrnnte and Chatm G d . Visiializing 3D fim. IEEE Compdm Gmphics and Applications, 18(4):49 53, 1998.

[XI] Victoria L. Interrante. illustrating surface shape in volume data via principal direction-driven 3D line integral convolution. In nimer Whitted, editor, SIG- GRAPH 97 Confemce Pmceedmgs, Annuai Conference Series, pages 109 116. M M SIGGRAPH, Addison Wesley, aug 1997.

[54] Jamee T. Kajiya and Brian P. Von Herzen. Ray tracing volume densities. In Hanlc Christiansen, editot, Computer Gmphics (SIGGRAPH '84 Proceedings), volumc 18, pagca lG5-174, July 1984.

[55] K. Kaugcrs and A. Brama. Catgraph: Visualizing largc lahclcci graphs. Tcch- nicai Report NMSU-TR-92-CS08, University of New Mexico, 1992.

(561 Timothy L. Kay and James T. Kajiya. Ray tracing complex scenes. Computer Gmphicu, 20(4):26+278, August 1986.

[57] T. Keahey and E. Robertson. Techniques for nonlinear rnagnification trans- formrrtiom. In S. Clud, S. Eick, and N. Cedam, trditoru, Proceedinp of the IEEE C o n f m c e on Infornation Vlsualization, pagcn 38 45, San Rancim, California, &t 1996. IEEE Computer Society Press.

[58] T. Keahey and E. Robertson. Nonlinear magaification fields. In J. DiIl and N. Gedam, editom, fhcedinga of the IEEE Confmnce on Information Vi- .9rrdizatimi, pages 51 58, Phoenix, Arizona, Oct 1N7. IEEE Gmputer Society Press.

f59] T. Alan Keabey. Vialiration of hi&-dimensional clusters using nonlinear magnification. Technical Report LA-UR-98-2776, 1.m Alamos Nationai Labratory, 1998.

[60] C. A. Kelsey. Tlie Perception of Viaud I n ~ o ~ o n , chapter Detection of visilal information, pages 30 51. Springer-Veslag, 1993.

[61] Y. Kurzion and R. Yagel. Space deformation using ray deflectors. In Proceedinga of the 6th Eurogmphics Workshop on Rendering, pages 21-32, June 1995.

[62] Y. Kunion lrnd R. Yagel. Conthoui a d dkontinuuiw deformution wing my detlectors. Zn PmceGdinp of CRAPHfCON'96, plrges 102-110, jul 1996.

(631 Yair Kurzion and b n i Yagel. interactive space deformation with hardware- amiriteci rendering. IEEE Compter Cmphia 8 Applicritiona, 17(5), 1997.

[64] Philippe Lacroute and Marc Levoy. Fast volume rendering using a shear-warp factoriastian af the viewing t do rmat ion . In Andrew Glmaer, editor, SIG- GRAPH '94 Conference Proeeeding19, Cornputer Graphia Pro<ieedingp, Anniial Conference Series, pages 431-458. ACM SIGGRAPH, .4CM Press, July 1994.

[65] John Lamping and Ramana Rao. Laying out and visuali7hg large trees using a hyperbolic space. In Pmceedinga O! the ACM Synposium on User Interface Softurate and Teehnology, Visualization 1, pages 13-14,1994.

[ôôj David Laur and Pat Hanrahan. Hierarchical splatting: A progremive refinement algorithm for volume rendering. Cornpulm Gmphics, 25(4):285 288, July 1991.

[67] Y. K. Leung and M . D. Appedey. A review and taxoaomy of distortion-oriented presentation techniques. ACM TOCHI, 1(2):126-160, 1994.

[68] Marc Levoy. Diplay of surla~es h m volume data. IEEE Computer Cmphtcrr and Applications, 8(3):29 37, May 1988.

[69] Marc Lewy. A hybrid ray tracer br rendering polygon and wlume data. IEEE Computer Gmphics and AppIhtioris, 10(2):33 40, Mar& 19M.

[70] William E. Loteosen and H a m y E. Che. Marching cubes: A high resolution 3D surfacc construction aigocithm. Cornputer Graphies, 21 (4):163-169, July 1987.

[71] Jock Mlrckiniy, Ben Shneidermiui, Colin Wm, W i i l h Wright, and Nohum Gershon. Keynote panel: 26 va. M. In John Dill and Nahum Gershon, d i - ton, Proceedings of the IEEE Cm#emœ on Injormation V h d h t r o n , page xi. IEEE Computer Wety Press, 1997.

[72] ,JO& D. Mackinlay, George G. Robertson, and Stuart K. Card. The perspective d l : Detail and contact smoothly integratd. In PrPEeeding of ACM CH1'91 Confemce on Human Factors in Computing Sptem, Idormation Visualiza- tion, 173-179,1991.

[73] Jock D. Mackinlay, George G. Robertson, and Robert DeLine. Developing calendar visualizers for the information v i s u d i . ln P d i n g s of the ACM

Symposium on User Interface Sojtwom and Teclrnology, VisuIilization II, pages 104-118,1994.

[74] Tom Malobender. Fourier volume rendering. ACM î h ~ o d i o w on Gmphics, 12(3):233-250, July 1993.

[75] Donald Mcagùcr. Gcomctric modcling using octrcc cncoding. Computet Gmph- iu a d Image Aucesjing, 19(2):129-147, June 1982.

[76] Brett Milash, Catherine Plaisant, and Anne Rose. Ufelines: Visualizing per- a o d hietories. la Pmcdiny of ACM CHI 96 C o n f m œ on Humon Factors in Computing Sgstem, volume 2 of VIDEOS: V~wlizoiion, pagies 392 393,1996.

[77] K. Misue and K. Sugiyama Multi-Mewpoint perspective diaplay methods: For- mulation and application to compound digraphs. In H. J. Bdlinger, editor, Humon Aspects in Computing: decrign and Use o j Intemctive Sgstems and In- formation Management, pqes 834 838. Elsevier Science Pu blishers, 1991.

[78] D. A. Mitra. A fisheye presentation strategy: AircraEt maintenance da ta In Human-Cornputer Intemction - INTERACT '90, pages 875-880, 1990.

[79] M. E. Newell, R. G. Newell, and T. L. Sancha. A solution to the hidden surface problem. In P d i n g s of the ACM Annuai Confmnce, volume 1, pages 443- 450, Won, Marrsschusetts, August 1972.

[80] E. b i k . A apace of pmntrtinn empha~ia techniques for vbualizing graphs. In P d n p of Gmphacs Interface 'gd, pages 225-233, May 1994.

[el] Jean Piaget. Piaget'~ theory. In P.H.Mmn, di tar , Canntchaeb Manud of CClüd Pqchdw. N.Y.:Wiiey, 1970.

[82] B. Preim and T. Strothotte. Annotating and interacting with 3D-models for explanatinn puqmm: Syritem mbiûxture. In G. P. Fmnti and T. Riut, 4- itom, Proceedinp of ECAI'96 Workhop TowaniR a Standard Retemce Mo& for Intelligent Multimedia h e n t o t i o n Systema. ECAP96 - J.von Xeumann Computer Society, Bathori u. 16, H-1054 Budapest, August 1996.

[83] Bernhard Preim, Andreas Raab, and Thomas Shothotte. Coherent aooming of illiurtratiom with 3D-graphies and text. In Grophim Inderface, pages 103 113, May 1997.

[841 Andseas Raab and Michael Rüger. YD-Zoom: Intemdive VUtlOIUation of Siruc- tum and Relotiom in Complez Gmphics, pages 125-132. Jnfix, Sankt Augustin, lm7.

[85] Fbmcuia Rw, and Stuart K. C d . The table lem: Merging graphical and symbolic repreaentations in an intemüve focus + context vidization for tabular information. In Beth Adelruin, Swan Dumah, and Judith Olson: editoni, Pm- ccedinp of the Confmnce on Hman Factors in Compuiing Syetems, pege~

318-322, New York, NY, USA, April 1994. ACM Press.

[86] Jun Rekimoto and Mark G m . The information cube: Using transperency in 36 informatioa visualization. in Thid A n n d Wonkdhop on Infomtion Technologies d S@ma (WITS'!&P', pqes 125 132,1993.

[87] Felix Ritter, Bernard Preim, Oliver Deusaen, and Thomas Strothotte. Using a 3d puzzle an a mtapbcir for learaing ~ p a t i d mlatio~lfi. In Cmphics Interfoce, pages 171 178, 2000.

[88] George Robertson, Mary Cmwinski, Kevin Ilanion, Daniel C. Robbms, David Thiel, anci Maarten van DantPch. Data Mountain: Using spatial memory for document management. In %inp of the 11th A n n d Symposium on User Interface Sojîware a d Technologie (UIST-98), pages 153 162, New York, Novcmhcr 1-4 1998. ACM Prcwi.

(891 Gmrge Robertson, Mary Czerwinuki, and Maarten WM Dantzich Immersion in daktop vistual d i t y . In h x d i n g s of the ACM Symposium on User Interface Softtuate and Tdnology, 3D Interaction Techniques, pages 11-19,1997.

[90] George Robertson, Maarten van Dantaich, Daniel Rohhins, Mary Czerwinski, Ken Hinkley, Kirsten Risden, navid Thiel, and Vadim Gorokhovsky. The task gallery: A 3D window manager. in Aoceedings of the ACM Confennce on Humon Factors in Computing System: CH. 2000, pages 494-501, 2000.

[91] George G. Rolmtson and Jock D. Biackiulay. The documeiit leus. in hceeùirrgi~ of the ACM Symposium on User In tedm Sofiwm and Technology, Vhaiizing information, pqpi 101-108,1993.

[92] George G. Robertson, Jock D. Mackinlay, and Stuart K. Card. Cone trees: Animated 3D visualisations of hierarcbid information. ln Proceedinga of ACM CHI'91 Conference on Human Faciors in Camputing Systems, Information Vi- sualization, pagee 18W94,1991.

[93] B. Rogers and M. Graùam. SUailarities between motion yarallax and stereopis in humrui depth perception. Vision Reseamh, 2261 270,1982.

[94] Manojit Sarkar and Steven P. Reiss. Mauiuplating screen space with stretch- tools: Visualking large structure on s m d screen. Technieal Report CS-92-42, Bmwn University, lm.

[95] Mmojit Sarkrir, Scott S. Snibbe, Oren J. TveMky, m d Steven P. Rek. Stretch- ing the nibber sheet: A metophor for vhaiizing large layouts on small screene. In hxeeàings of the ACM Sympo8ium on User Interface Software and Tech- nol49~~, Vieualiaing Information, pgm 81 91, 1993.

[96] D.S. Schlusselberg, K. Smith, and D.J. Woodward. Three-dimensional display of medicd image voliimes. In h m e d i n p of NCGA'86 Conference, volume 3, pages 114 123. National Computer Graphics Association, 1986.

[97] Peter Shirley and Allan Tuchman. A polygonal appmximation to direct scalar volume rendering. In Computer Cmphics (San Diego Workshop on Volume VUwlizoiion), volume 24-5, plrges 63 70, November 1990.

[98] Ben Sbneideman. The eyes have it: A task by data type taxonomy for information visualizations. In Pmœedings of the iEEE Symposium on Viswl Langwges, pagea 336 343, Washington, September 3 6 1996. IEEE Computer Socicty Prcss.

1991 R Spcncc and M. Appcrly. Data hasc navigation: an officc cnviromcnt for thc profissional. Behaviour and Information Technology, 1(1):43 54, 1982.

[lCK)] J. Stok and J.J. van Wijk. Surface-particles for .PD Flow Vkdization, pages 114-130. Springer-Verlag. Advances in Scientific Visualizatioa, 1992.

[lQl] M. A. Storey and H. A. Müller. Graph layout adjustment strategies. In Cmph hwhg '95, pagw 487 499,199û.

Cl021 Mark Tiges, M. S. T. Carpenclaie, and Briau Wyvill. Geueralized distarice metnm for implicit ~ucim mdel'ig. In Proceedings of the Tmih Westem Computer Gmphics Sppoaium, Mar& 1999.

[103] Edward R. 'hftc!. The Vieuol Ouplay of Qwntitotive Infomtion Gmphim P m , Cheshire, Connecticut, U.S.A., 1983.

(1041 E d d R M e . EnMsioning Information Graphies Press, Cheshire, Con- necticut, U.S.A., May 1990.

[IO51 Edwerd R. TuRe. Viawl Explandions: Images ond Quantiiies, Euidence and Normtive. Graphics Press, Bax 430, Cheshire, CT 06410, USA, 1997-

[lûô] H. K n iy and L. T. niy. Direct 2-D &play of 3-D objets. IEEE Cornputer Cmphic.9 and Appliwtion.9, 4(10):29 33, October 1984.

(1071 Cees van krkel, David W. Parker, and Antony R Fmnklin. Multiview 3d- Icd. In SPIE: intendional confennce on EIectronic Imaging, San J e , January 1996.

[10S] Jarke J. van Wik. A ranter graphicn approach t o flaw halization. in C. E. Van- doni and D. A. Duce, editors, Eumgmphics '90, pages 251-259. North-Holland, Septemher 1990.

[109] J.J. van Wijk. Ibndering liena on c d surf-. In Pmr .ng , r of the Etm- gmphics W ~ h h o p on Visualkation in SMentific Computing. Springer-Verlag, 1990.

[110] Sutya Vanka and David Klein. Colortool: An information tool for crosgcultural design. In PfOceedings of the Human Factors and Etgonomics Society 39th Annud Meeting, volume 1 of CONSUMER PRODUCTS: CWune, Pmeption, and People in Product Design [Lecturel, pages 341 345,1995.

[Ill] Ham Wallach and D.N. O'Comell. The Linetic depth effect . Journd 4 Ezpet- imentai Psgtchology, 45(4):205217, April 1953.

[112] C. Ware and G. h c k . Viewing a graph in a virtual reality display ie three times aa good as a 2D diagram. In IEEE Confmce on Viswl Languages, pagea 182-183, O c ~ h t 1994.

[113] Colin Ware. Infonndion Visualization, Perrception for Deaign. Morgan Kauf- muun, 20.

[Il41 Coliri Ware, Kevin W. Arthur, and Kellogg S. Booth. Fisb tank virtual r d - ity. In Stacy hhlund, K. Mullet, A. Hendemn, E. Hohagel, and T. White, editors, Fm. ACM Conf. Human Factors in Coniputirig Systems, INTERCHI (INTERACT 8 CHI), pages 37 42,24 29 April 1993.

[Il51 Colin Wm, David Hui, and Glenn Franck. Visuaiiaiog abject orientmi mft- ware in three dimensions. In koc. IBM Ce* /or Aduanceû Sitrdiea Conf., CASCON, October 1993.

[ I l4 A. Watt and M. Watt. Advanad animation and nndenng technique,* - Themy and pmctice. Addimn-Wesley, New York, 1992.

[117] Lee Westover. Footpriat evaluation for volume tendering. b Forest Baakett, editor, Cornputer Graphies, volume 24, pages 367 376, August 1990.

[118] k Alm Wtnitowr. SPLATTING: A prrrallel, f e e d - t o r d volume rendering algorithm. Techaical Report TR91-029, Department of Cornputer Science, Uni- versity of North Ciuolina - Chape1 Hill, jul1991.

[119] Chrintopher D. Wickem, David H. h1erwin, and Emilie L. Lin. Implicrrtions of graphics enhancements for the visualization of scienticfic data: Dimensional integrality, atempis, motion and mesh. In hmdingm of the ,?&h Annud Meeting of the Human Factors Society, pages 44 61, 1994.

[120] Orion Wilson, Allen VanGdder, and Jane Wilhelms. DIRECT VOLUME RJ3X DERING VIA 3D TEXTURES. Technical Report CCSC-CRG94-19, Cniversity of California, Santa Cruz, Jack Baskin School of Engineering, .lune 1994.

[121] Ulrih Wiss and David Cm. A cognitive classification fiamework for 3- dimensional information visualization. Technical Report LW-TR-199814, Luleâ University of Technology, 1W.

[122] Shumin Zhai, William Bwton, and Paul Milgram. The "silk cursor": Inmti- gating transparcncy for 3D targct acquisition. In P d i n g s of ACM CHI'94 Conjznme on Human Factors in Computing Systems, volume 2 of PAPER ABSTRACTS: Intemcting in 3-9-0, page 233, 1994.

11231 Shumin Zhai, Willirun Buxton, and Paul Milgram. The "silk cursor": Investi- gating twpiut!ncy for 3D target acquisition. In Proceedinga of ACM CHl'gd Confemce on Human Factors in Computing Syatema, volume 1 of Intemcting in Y-D, pages 459 464, 1994.

[124] Shumin Zhu, William Buxton, and Paul Milgrm. The partid-ocdusion dwt: Utilizing semitransparency in 3D humaniiomputer interaction. ACM Ziurisac- tioru on Cornputer-Humon Intmtion, 3(3):254 284, lm.

· occlusion resolution operators for threedimensional detail-in-context david j. cowperthwaite...

Documents