Virtual Worlds Lab

Testbed for Mobile Augmented Testbed for Mobile Augmented Battlefield VisualizationBattlefield Visualization

September, 2003September, 2003

William Ribarsky and Nickolas FaustGVU Center and GIS Center

Georgia Institute of Technology

Virtual Worlds Lab

Matrix of Proposed Activities and ResultsProposedTasks

What Was Done Done This Year Future Work

Multimodal3D Interaction

Development andevaluation of gestureand voice interface;testing of otherinterfaces

Implementation and use ofnew interface in mobileenvironment

MobileVisualization

Define and initialimplementation ofmobile situationalvisualization

More completeimplementation,evaluation, and use

Extensive use andsharing of system,data

4D Modeling Initial automated treemodeling

Modeling large collectionsof accurately placedbuildings

Joint multisourcemodeling efforts

Dynamic,UniversalDataStructures

Initial paged object datastructure for simplebuildings

Dynamic, scalablestructure for handlingobjects from multiplesources

Completescalability withefficient access

InteractiveRenderingandVisualization

Initial work on simplebuilding LODs andintegration with terrain,texture

New view-dependent,appearance preservingrendering techniques

New techniques forlarge collections ofobjects and newmanipulation

CollaborationandIntegration

Development of initialLOD approach. Flexibledevelopment of VGISenvironment

Initial implementation of atestbed for mobileaugmented

Multilevelintegration andsharing

TechnologyTransfer

Work with Army,Sarnoff Corp., NRL

Homeland Defense andother outreach efforts

Army, DARPA,Homeland Defense

Virtual Worlds Lab

Mobile Situational VisualizationAn extension of situation awareness that exploits and integrates

interactive visualization, mobile computing, wireless networking, and multiple sensors:

• Mobile users with GPS, orientation sensing, cameras, wireless• User carries own 3D database• Servers that store and disseminate information from/to multiple

clients (location, object/event, weather/NBC servers)• Location server to manage communications between users and

areas of interest for both servers and users• Ability to see weather, chem/bio clouds, and positions of other

users• Accurate overviews of terrain with accurately placed 3D buildings• Ability to mark, annotate, and share positions, directions, speed,

and uncertainties of moving vehicles or people• Ability to access and playback histories of movement• Placement of multiresolution models from MURI team members into

environment

Results of real-time collection of GPS path at night (left); screen shot with annotated path in red (right).

New lightweight wearable system

GPS and orientation tracker

Virtual Worlds Lab

Weather/Atmospheric Server

Annotation ServerAnnotated views with updated user location and orientation

Spread of dynamic Sarin gas cloud with positions of first responders

Mobile Situational Visualization

Accurate Shared Locations

Virtual Worlds Lab

Mobile Situational Visualization System

Drawing Area

Buttons Pen Tool

Mobile Team

Collaboration Example

collaborators

Shared observations of vehicle location, direction, speed

Virtual Worlds Lab

Collaborative Environment

• Everybody has a location in space and time in the Virtual World

• Geographic server lookup approach– Users– Location Servers– Data Servers

Weather Server

UserUser

User

Location Server

Traffic ServerAnnotation Server

GeoData Server

Virtual Worlds Lab

• Everybody has a location in space and time in the Virtual World

• Geographic server lookup approach– Users– Location Servers– Data Servers

UserUser

User

Location Server

Traffic ServerAnnotation Server

GeoData Server

Weather Server

Collaborative Environment

Virtual Worlds Lab

Mobile Situational Visualization

Video

Virtual Worlds Lab

What is Novel and Compelling AboutMobile Situational Visualization?

Mobile battlefield visualization was an original proposed (and accepted) task. That’s pretty compelling!

But, beyond thatInstant placement of environmental activity information within the geospatial environment combined with fast sharing and use are novel and compelling.

-Fast, accurate, and specific annotation of activity information (both user-controlled and automated logging)

-Immediate updates of databases with this information-Server structure for sharing this with collaborators or commanders in the area of interest

-Use in computations and simulations (some launched automatically)

Virtual Worlds Lab

Matrix of Proposed Activities and ResultsProposedTasks

What Was Done Done This Year Future Work

Multimodal3D Interaction

Development andevaluation of gestureand voice interface;testing of otherinterfaces

Implementation and use ofnew interface in mobileenvironment

MobileVisualization

Define and initialimplementation ofmobile situationalvisualization

More completeimplementation,evaluation, and use

Extensive use andsharing of system,data

4D Modeling Initial automated treemodeling

Modeling large collectionsof accurately placedbuildings

Joint multisourcemodeling efforts

Dynamic,UniversalDataStructures

Initial paged object datastructure for simplebuildings

Dynamic, scalablestructure for handlingobjects from multiplesources

Completescalability withefficient access

InteractiveRenderingandVisualization

Initial work on simplebuilding LODs andintegration with terrain,texture

New view-dependent,appearance preservingrendering techniques

New techniques forlarge collections ofobjects and newmanipulation

CollaborationandIntegration

Development of initialLOD approach. Flexibledevelopment of VGISenvironment

Initial implementation of atestbed for mobileaugmented

Multilevelintegration andsharing

TechnologyTransfer

Work with Army,Sarnoff Corp., NRL

Homeland Defense andother outreach efforts

Army, DARPA,Homeland Defense

Virtual Worlds Lab

Integrated, Comprehensive Modeling

•To build comprehensive models, we need a range of modeling techniques.

•We also should combine techniques for richer and more complete models.

Model Detail

Geo

-acc

urac

y

Low Mid High

Low

Mid

Hig

h

Georgia Tech

Thousands to tens of thousands of buildings and treesHundreds of semi-automatically modeled buildings

USCTens to Hundreds of semi-automatically modeled buildings

BerkeleyHundreds of automatically modeled buildings

Virtual Worlds Lab

Integrated, Comprehensive Modeling

•To build comprehensive models, we need a range of modeling techniques.

•We also should combine techniques for richer and more complete models.

Detail

Geo

-acc

urac

y

Low Mid High

Low

Mid

Hig

h

Integrated, comprehensive models with combined techniques

Virtual Worlds Lab

New Results on Modeling Large Collections

•Generic models extruded from accurate footprints with accurate locations. (11,000 automatically generated from insurance GIS databases).

-Complete models with roofs-Generic façade textures-Databases available for automatically generating whole city (hundreds of thousands)

•Automatic generation of accurately located tree models (thousands) from high-res imagery.

•Creation of hundreds of specific buildings using commercial or self-developed (semi-automatic) software.

(individual 3D buildings have brown roofs)

3D CAD modeled objects on high resolution terrain

Virtual Worlds Lab

Automatic Identification and Placementof Trees, Shrubs, and Foliage

This can be used with Ulrich Neuman’s or Avideh Zakhor’s results to automatically identify, remove, and model foliage.

Virtual Worlds Lab

Automated identification and modeling of trees

Application to Tree Modeling

Accurate placement of 3D modeled trees

Virtual Worlds Lab

Matrix of Proposed Activities and ResultsProposedTasks

What Was Done Done This Year Future Work

Multimodal3D Interaction

Development andevaluation of gestureand voice interface;testing of otherinterfaces

Implementation and use ofnew interface in mobileenvironment

MobileVisualization

Define and initialimplementation ofmobile situationalvisualization

More completeimplementation,evaluation, and use

Extensive use andsharing of system,data

4D Modeling Initial automated treemodeling

Modeling large collectionsof accurately placedbuildings

Joint multisourcemodeling efforts

Dynamic,UniversalDataStructures

Initial paged object datastructure for simplebuildings

Dynamic, scalablestructure for handlingobjects from multiplesources

Completescalability withefficient access

InteractiveRenderingandVisualization

Initial work on simplebuilding LODs andintegration with terrain,texture

New view-dependent,appearance preservingrendering techniques

New techniques forlarge collections ofobjects and newmanipulation

CollaborationandIntegration

Development of initialLOD approach. Flexibledevelopment of VGISenvironment

Initial implementation of atestbed for mobileaugmented

Multilevelintegration andsharing

TechnologyTransfer

Work with Army,Sarnoff Corp., NRL

Homeland Defense andother outreach efforts

Army, DARPA,Homeland Defense

Virtual Worlds Lab

Organizing Large Collections of 3D Models for Interactive Display

•Merging of different types and formats•Automated replacement of structures for overlapping areas

Common format and organization for different types

Q

Q QQ Q

Q

Q QQ Q

Q

Q QQ Q

Q QQ Q

Linked Global Quadtrees

Virtual Worlds Lab

Paging, Culling, and Fast Rendering

Quadcell

Block

Block

QQQQ

Linked global quadtree

Block

BlockBlock

Block

Out-of core Storage

Virtual Worlds Lab

Integrated, Interactive Visualization ofLarge Collections of Models

Video

Virtual Worlds Lab

Matrix of Proposed Activities and ResultsProposedTasks

What Was Done Done This Year Future Work

Multimodal3D Interaction

Development andevaluation of gestureand voice interface;testing of otherinterfaces

Implementation and use ofnew interface in mobileenvironment

MobileVisualization

Define and initialimplementation ofmobile situationalvisualization

More completeimplementation,evaluation, and use

Extensive use andsharing of system,data

4D Modeling Initial automated treemodeling

Modeling large collectionsof accurately placedbuildings

Joint multisourcemodeling efforts

Dynamic,UniversalDataStructures

Initial paged object datastructure for simplebuildings

Dynamic, scalablestructure for handlingobjects from multiplesources

Completescalability withefficient access

InteractiveRenderingandVisualization

Initial work on simplebuilding LODs andintegration with terrain,texture

New view-dependent,appearance preservingrendering techniques

New techniques forlarge collections ofobjects and newmanipulation

CollaborationandIntegration

Development of initialLOD approach. Flexibledevelopment of VGISenvironment

Initial implementation of atestbed for mobileaugmented

Multilevelintegration andsharing

TechnologyTransfer

Work with Army,Sarnoff Corp., NRL

Homeland Defense andother outreach efforts

Army, DARPA,Homeland Defense

Virtual Worlds Lab

Handling Complicated Models

Bounding box

Selected LOD

View-Dependent LOD for large collections of complicated models

Q

Q QQ Q

Q

Q QQ Q

Q

Q QQ Q

Q

N LevelsLinked Global Quadtrees

Results of view-dependent simplification. The blue box is the viewing window; fully textured models with and without meshes displayed are shown on the left and right, respectively. (Top) Full resolution mesh and textures within the window. (Bottom) Significantly reduced resolution mesh and textures within the window without reduction in visual quality.

Viewpoint

Virtual Worlds Lab

Quadric Error Approach to Simplification

•Initial development Garland and Heckbert, SIGGRAPH, 1997•Quadric approach yields “optimal” simplification by permitting generalized contractions between vertices and keeping track of the deviation from the original mesh

v1 v2

contractionv

)v(

vQvT Use quadric matrix to find a vertex with error within ε; Δ is the surface at error

value ε.

v1 v2

generalcontraction

Non-topological simplification

v

Virtual Worlds Lab

Limitations on Basic Quadric Approach

•No concept of view-dependence and continuous LOD•No structure for large collections of objects•Geometry error metric; no appearance-preserving metric (e.g., for textures, shading, lighting). A combined metric is best.

full resolution w/o appearance metric

with appearance metric

Application of appearance-preserving metric to a textured object (Cohen et al., SIGGRAPH 98)

[ ]

Virtual Worlds Lab

View-Dependent Continuous LOD Tree

The vertex front is circled. Green nodes are active-interior, blue nodes are active-boundary, and orange nodes are inactive. Here, vertex V7 is split and vertices V10 and V11 are merged.

V10 V11

V5

V8 V9

V4

V2

V1

V14 V15

V7V6

V3

Simplification Pass:split: V7; merge: V10, V11

V10 V11

V5

V8 V9

V4

V2

V1

V14 V15

V7V6

V3

V10V11

V14

V15V8 V9

V6

V4

V10V11

V7

V6

V4

V5

V14

V15

V6

(a)

(b) (c)

Simp Pass

The pink, purple, and dark gray triangles are subfaces of V7, V5, and V4, respectively. (a) Full mesh. (b) Tree on left. (c) Tree on right.

Block

Façade 1 …

Façade N

LOD Hierarchy

…

…

…

Object M

…

…

…

Object 1

…

…

…

…

…

…

…

QQQQ

Linked global quadtree

Virtual Worlds Lab

View-Dependent Appearance-Preserving Simplification

Collapse Distance Deviation

possible surface Mi

current surface Mi-1

original surface M0

deviation vectors

Va Vb

Vc

PO

PC

Two-Way Incremental Distance Deviation

Va Vb

Vc

(A)

(B)

Quadric Error Deviation

Va Vb

Vc

(C)

Va Vb

Vc

P

C

One-Way Incremental Texture Deviation

(D)

Va Vb

Vc

P

A

Two-Way Incremental Texture Deviation

P

BP

C

(E)

(F)

V

a

V

b

V

c

P

O

Total Texture Deviation

Virtual Worlds Lab

View-Dependent Appearance-Preserving Simplification

Video

Virtual Worlds Lab

Matrix of Proposed Activities and ResultsProposedTasks

What Was Done Done This Year Future Work

Multimodal3D Interaction

Development andevaluation of gestureand voice interface;testing of otherinterfaces

Implementation and use ofnew interface in mobileenvironment

MobileVisualization

Define and initialimplementation ofmobile situationalvisualization

More completeimplementation,evaluation, and use

Extensive use andsharing of system,data

4D Modeling Initial automated treemodeling

Modeling large collectionsof accurately placedbuildings

Joint multisourcemodeling efforts

Dynamic,UniversalDataStructures

Initial paged object datastructure for simplebuildings

Dynamic, scalablestructure for handlingobjects from multiplesources

Completescalability withefficient access

InteractiveRenderingandVisualization

Initial work on simplebuilding LODs andintegration with terrain,texture

New view-dependent,appearance preservingrendering techniques

New techniques forlarge collections ofobjects and newmanipulation

CollaborationandIntegration

Development of initialLOD approach. Flexibledevelopment of VGISenvironment

Initial implementation of atestbed for mobileaugmented

Multilevelintegration andsharing

TechnologyTransfer

Work with Army,Sarnoff Corp., NRL

Homeland Defense andother outreach efforts

Army, DARPA,Homeland Defense

Virtual Worlds Lab

Implementing and Using the Testbed

•Merging of tens of thousands (and more) of models from multiple sources.

•Efficient organization and culling of massive collections of 3D objects.

•Integration of view-dependent methods for accurate and efficient display of complex models.

•Deployment and use of mobile situational visualization capability.

Virtual Worlds Lab

Technology Transfer

•The VGIS visualization system with capabilities developed here (including mobile visualization) was a key part of the Georgia Tech Homeland Defense Workshop and will be part of the GT Homeland Defense Initiative with support at the State and National levels.

•The system is being used as part of the Sarnoff Raptor system, which is deployed to the Army and other military entities. In addition our visualization system is being used as part of the Raptor system at Scott Air Force Base.

•We are in discussion with the Department of the Interior on use of our mobile situational visualization capability to develop Anytime-Anywhere information system resource accessibility for countering asymmetric threats.

Virtual Worlds Lab

Plans for Next Year

•Full deployment of mobile situational visualization capability with sharing of the system and the results with team members.

•Further development of automated model building from multisource data. This will be a collaborative effort with other team members. We will move towards a robust system with ability to merge and increment model sets and update models (adding improvements to make generic models more detailed and specific as data are available).

•Development of fully scalable 3D object organization and interactive visualization capability extending to hundreds of thousands of accurately located buildings and trees (or more).

•Full integration of view-dependent capability for complex models.