collaborative mixed reality visualization of an archaeological excavation hrvoje benko department of...
TRANSCRIPT
![Page 1: Collaborative Mixed Reality Visualization of an Archaeological Excavation Hrvoje Benko Department of Computer Science Columbia University MIT talk – January](https://reader035.vdocuments.mx/reader035/viewer/2022070305/551477e0550346b2598b4622/html5/thumbnails/1.jpg)
Collaborative Mixed RealityCollaborative Mixed Reality Visualization of an Archaeological Visualization of an Archaeological
ExcavationExcavation
Hrvoje BenkoHrvoje BenkoDepartment of Computer ScienceDepartment of Computer Science
Columbia UniversityColumbia University
MIT talk – January 12, 2005MIT talk – January 12, 2005
![Page 2: Collaborative Mixed Reality Visualization of an Archaeological Excavation Hrvoje Benko Department of Computer Science Columbia University MIT talk – January](https://reader035.vdocuments.mx/reader035/viewer/2022070305/551477e0550346b2598b4622/html5/thumbnails/2.jpg)
2
Multidisciplinary TeamMultidisciplinary Team
![Page 3: Collaborative Mixed Reality Visualization of an Archaeological Excavation Hrvoje Benko Department of Computer Science Columbia University MIT talk – January](https://reader035.vdocuments.mx/reader035/viewer/2022070305/551477e0550346b2598b4622/html5/thumbnails/3.jpg)
3
MotivationMotivation
►Excavation is destructive and Excavation is destructive and physically “unreconstructable” processphysically “unreconstructable” process
►Need to preserve as much data as Need to preserve as much data as possible for analysispossible for analysis
►Data interpretation happens off-siteData interpretation happens off-site►Current tools focus on 2D data and do Current tools focus on 2D data and do
not incorporate 3D informationnot incorporate 3D information►Many experts—collaboration is a must!Many experts—collaboration is a must!
![Page 4: Collaborative Mixed Reality Visualization of an Archaeological Excavation Hrvoje Benko Department of Computer Science Columbia University MIT talk – January](https://reader035.vdocuments.mx/reader035/viewer/2022070305/551477e0550346b2598b4622/html5/thumbnails/4.jpg)
4
Archaeological Excavation at Archaeological Excavation at Monte Polizzo, Sicily, Summer Monte Polizzo, Sicily, Summer
20032003Ian Morris, Director (Stanford University)Ian Morris, Director (Stanford University)
![Page 5: Collaborative Mixed Reality Visualization of an Archaeological Excavation Hrvoje Benko Department of Computer Science Columbia University MIT talk – January](https://reader035.vdocuments.mx/reader035/viewer/2022070305/551477e0550346b2598b4622/html5/thumbnails/5.jpg)
5
Working in the field!Working in the field!
![Page 6: Collaborative Mixed Reality Visualization of an Archaeological Excavation Hrvoje Benko Department of Computer Science Columbia University MIT talk – January](https://reader035.vdocuments.mx/reader035/viewer/2022070305/551477e0550346b2598b4622/html5/thumbnails/6.jpg)
6
Meshed 3D Model (13 scans)Meshed 3D Model (13 scans)
![Page 7: Collaborative Mixed Reality Visualization of an Archaeological Excavation Hrvoje Benko Department of Computer Science Columbia University MIT talk – January](https://reader035.vdocuments.mx/reader035/viewer/2022070305/551477e0550346b2598b4622/html5/thumbnails/7.jpg)
7
We have a 3D site model, We have a 3D site model, now what?now what?
RealReal VirtualVirtual
![Page 8: Collaborative Mixed Reality Visualization of an Archaeological Excavation Hrvoje Benko Department of Computer Science Columbia University MIT talk – January](https://reader035.vdocuments.mx/reader035/viewer/2022070305/551477e0550346b2598b4622/html5/thumbnails/8.jpg)
8
High Resolution Images
DrawingsField Notes
Panoramic Images
Videos
GIS Data
![Page 9: Collaborative Mixed Reality Visualization of an Archaeological Excavation Hrvoje Benko Department of Computer Science Columbia University MIT talk – January](https://reader035.vdocuments.mx/reader035/viewer/2022070305/551477e0550346b2598b4622/html5/thumbnails/9.jpg)
9
High Resolution Images
DrawingsField Notes
Panoramic Images3D Object Models
3D Site Model Videos
GIS Data
Database
![Page 10: Collaborative Mixed Reality Visualization of an Archaeological Excavation Hrvoje Benko Department of Computer Science Columbia University MIT talk – January](https://reader035.vdocuments.mx/reader035/viewer/2022070305/551477e0550346b2598b4622/html5/thumbnails/10.jpg)
10
Two ProblemsTwo Problems
►How to combine all this data in one How to combine all this data in one seamless environment?seamless environment?
►How to make it easy to interact with?How to make it easy to interact with?
![Page 11: Collaborative Mixed Reality Visualization of an Archaeological Excavation Hrvoje Benko Department of Computer Science Columbia University MIT talk – January](https://reader035.vdocuments.mx/reader035/viewer/2022070305/551477e0550346b2598b4622/html5/thumbnails/11.jpg)
11
VITA:VITA: Visual Visual InteractionInteraction Tool Tool forfor
ArchaeologyArchaeology► Multiple usersMultiple users► Multiple displaysMultiple displays
Projected tabletopProjected tabletop HandheldHandheld High-resolution High-resolution
monitormonitor See-through head-See-through head-
wornworn► Multiple interaction Multiple interaction
devicesdevices MERL DiamondTouch MERL DiamondTouch
tabletable EssentialReality P5 EssentialReality P5
glovesgloves Speech inputSpeech input 6DOF tracker6DOF tracker
![Page 12: Collaborative Mixed Reality Visualization of an Archaeological Excavation Hrvoje Benko Department of Computer Science Columbia University MIT talk – January](https://reader035.vdocuments.mx/reader035/viewer/2022070305/551477e0550346b2598b4622/html5/thumbnails/12.jpg)
12
Design ConsiderationsDesign Considerations
►Use the most appropriate display for Use the most appropriate display for the given datathe given data
►Facilitate both human-system and Facilitate both human-system and human-human interactionhuman-human interaction
![Page 13: Collaborative Mixed Reality Visualization of an Archaeological Excavation Hrvoje Benko Department of Computer Science Columbia University MIT talk – January](https://reader035.vdocuments.mx/reader035/viewer/2022070305/551477e0550346b2598b4622/html5/thumbnails/13.jpg)
13
Modular ArchitectureModular Architecture
AR Module
DT ModuleSCREEN Module
HANDHELDModule
AR Module …
DatabaseMessageFacilitator
2D Modules
3D Modules
![Page 14: Collaborative Mixed Reality Visualization of an Archaeological Excavation Hrvoje Benko Department of Computer Science Columbia University MIT talk – January](https://reader035.vdocuments.mx/reader035/viewer/2022070305/551477e0550346b2598b4622/html5/thumbnails/14.jpg)
14
AR Module ComponentsAR Module Components
Head TrackerHand Tracker
(Intersense IS900)
Sony Head-Worn Display(LDI-D100B)
Microphone
P5 Glove(Essential Reality)
DiamondTouch Table Connector
(MERL)
![Page 15: Collaborative Mixed Reality Visualization of an Archaeological Excavation Hrvoje Benko Department of Computer Science Columbia University MIT talk – January](https://reader035.vdocuments.mx/reader035/viewer/2022070305/551477e0550346b2598b4622/html5/thumbnails/15.jpg)
15
Life-size Immersive Life-size Immersive ExplorationExploration
![Page 16: Collaborative Mixed Reality Visualization of an Archaeological Excavation Hrvoje Benko Department of Computer Science Columbia University MIT talk – January](https://reader035.vdocuments.mx/reader035/viewer/2022070305/551477e0550346b2598b4622/html5/thumbnails/16.jpg)
16
3D Multimodal Interaction3D Multimodal Interaction
► Provide natural interaction Provide natural interaction mechanism for our 3D mechanism for our 3D environmentenvironment
► ModalitiesModalities Speech: IBM ViaVoice 10Speech: IBM ViaVoice 10 Gestures: Gestures:
EssentialReality P5 gloveEssentialReality P5 glove Selection statistics: Selection statistics:
SenseShapesSenseShapes
► Focus on selectionFocus on selection Based on collaboration with Phil Based on collaboration with Phil
Cohen et al. (Cohen et al. (ICMI 2003ICMI 2003) and ) and SenseShapes (SenseShapes (ISMAR 2003ISMAR 2003) )
![Page 17: Collaborative Mixed Reality Visualization of an Archaeological Excavation Hrvoje Benko Department of Computer Science Columbia University MIT talk – January](https://reader035.vdocuments.mx/reader035/viewer/2022070305/551477e0550346b2598b4622/html5/thumbnails/17.jpg)
17
VirtualTrayVirtualTray
![Page 18: Collaborative Mixed Reality Visualization of an Archaeological Excavation Hrvoje Benko Department of Computer Science Columbia University MIT talk – January](https://reader035.vdocuments.mx/reader035/viewer/2022070305/551477e0550346b2598b4622/html5/thumbnails/18.jpg)
18
Desktop ComponentsDesktop Components
Touch-sensitive Projective Display
(MERL DiamondTouch)
High-resolutionDisplay
Handheld Display
![Page 19: Collaborative Mixed Reality Visualization of an Archaeological Excavation Hrvoje Benko Department of Computer Science Columbia University MIT talk – January](https://reader035.vdocuments.mx/reader035/viewer/2022070305/551477e0550346b2598b4622/html5/thumbnails/19.jpg)
19
World-In-MiniatureWorld-In-Miniature
![Page 20: Collaborative Mixed Reality Visualization of an Archaeological Excavation Hrvoje Benko Department of Computer Science Columbia University MIT talk – January](https://reader035.vdocuments.mx/reader035/viewer/2022070305/551477e0550346b2598b4622/html5/thumbnails/20.jpg)
20
Harris MatrixHarris Matrix
![Page 21: Collaborative Mixed Reality Visualization of an Archaeological Excavation Hrvoje Benko Department of Computer Science Columbia University MIT talk – January](https://reader035.vdocuments.mx/reader035/viewer/2022070305/551477e0550346b2598b4622/html5/thumbnails/21.jpg)
21
Enhanced Harris Matrix Enhanced Harris Matrix
![Page 22: Collaborative Mixed Reality Visualization of an Archaeological Excavation Hrvoje Benko Department of Computer Science Columbia University MIT talk – January](https://reader035.vdocuments.mx/reader035/viewer/2022070305/551477e0550346b2598b4622/html5/thumbnails/22.jpg)
22
Enhanced Harris MatrixEnhanced Harris Matrix
![Page 23: Collaborative Mixed Reality Visualization of an Archaeological Excavation Hrvoje Benko Department of Computer Science Columbia University MIT talk – January](https://reader035.vdocuments.mx/reader035/viewer/2022070305/551477e0550346b2598b4622/html5/thumbnails/23.jpg)
23
Cross-Dimensional Hybrid Cross-Dimensional Hybrid GesturesGestures
►Synchronized 2D and 3D gesturesSynchronized 2D and 3D gestures►Facilitate seamless transition across Facilitate seamless transition across
dimensionsdimensions
To appear in IEEE VR 2005
![Page 24: Collaborative Mixed Reality Visualization of an Archaeological Excavation Hrvoje Benko Department of Computer Science Columbia University MIT talk – January](https://reader035.vdocuments.mx/reader035/viewer/2022070305/551477e0550346b2598b4622/html5/thumbnails/24.jpg)
24
Cross-Dimensional Hybrid Cross-Dimensional Hybrid GesturesGestures
PullPush
To appear in IEEE VR 2005
![Page 25: Collaborative Mixed Reality Visualization of an Archaeological Excavation Hrvoje Benko Department of Computer Science Columbia University MIT talk – January](https://reader035.vdocuments.mx/reader035/viewer/2022070305/551477e0550346b2598b4622/html5/thumbnails/25.jpg)
25
Cross-Dimensional Hybrid Cross-Dimensional Hybrid GesturesGestures
PullPinDragRotate Push
To appear in IEEE VR 2005
![Page 26: Collaborative Mixed Reality Visualization of an Archaeological Excavation Hrvoje Benko Department of Computer Science Columbia University MIT talk – January](https://reader035.vdocuments.mx/reader035/viewer/2022070305/551477e0550346b2598b4622/html5/thumbnails/26.jpg)
26
Cross-Dimensional Hybrid Cross-Dimensional Hybrid GesturesGestures
PullConnectScaleDisconnect Push
To appear in IEEE VR 2005
![Page 27: Collaborative Mixed Reality Visualization of an Archaeological Excavation Hrvoje Benko Department of Computer Science Columbia University MIT talk – January](https://reader035.vdocuments.mx/reader035/viewer/2022070305/551477e0550346b2598b4622/html5/thumbnails/27.jpg)
27
Handheld Focus-in-Context Handheld Focus-in-Context DisplayDisplay
►Movable high-Movable high-resolution insetresolution inset Tracked by Tracked by
DiamondTouchDiamondTouch Projection Projection
suppressed in its suppressed in its boundsbounds
Physical magic Physical magic lenslens
![Page 28: Collaborative Mixed Reality Visualization of an Archaeological Excavation Hrvoje Benko Department of Computer Science Columbia University MIT talk – January](https://reader035.vdocuments.mx/reader035/viewer/2022070305/551477e0550346b2598b4622/html5/thumbnails/28.jpg)
28
Tabletop InteractionTabletop Interaction
![Page 29: Collaborative Mixed Reality Visualization of an Archaeological Excavation Hrvoje Benko Department of Computer Science Columbia University MIT talk – January](https://reader035.vdocuments.mx/reader035/viewer/2022070305/551477e0550346b2598b4622/html5/thumbnails/29.jpg)
29
User FeedbackUser Feedback
►Overall very positive reactionOverall very positive reaction►Archaeologists benefited from:Archaeologists benefited from:
Temporal–Spatial connectionTemporal–Spatial connection Aggregated collection of all dataAggregated collection of all data Accurate 3D modelAccurate 3D model Simple touch-based interactions Simple touch-based interactions
►Potential for increased collaborationPotential for increased collaboration
![Page 30: Collaborative Mixed Reality Visualization of an Archaeological Excavation Hrvoje Benko Department of Computer Science Columbia University MIT talk – January](https://reader035.vdocuments.mx/reader035/viewer/2022070305/551477e0550346b2598b4622/html5/thumbnails/30.jpg)
30
Room for ImprovementRoom for Improvement► Reduce wiresReduce wires► Reduce weight Reduce weight ► Eye occlusion hinders communication Eye occlusion hinders communication ► Missing data:Missing data:
More objects, features, notes and picturesMore objects, features, notes and pictures More scans during excavation (time-lapse spatial More scans during excavation (time-lapse spatial
record) record) ► Missing features:Missing features:
Virtual scale measure (implemented since)Virtual scale measure (implemented since) Variable site model scalingVariable site model scaling Improved selection in world-in-miniatureImproved selection in world-in-miniature
![Page 31: Collaborative Mixed Reality Visualization of an Archaeological Excavation Hrvoje Benko Department of Computer Science Columbia University MIT talk – January](https://reader035.vdocuments.mx/reader035/viewer/2022070305/551477e0550346b2598b4622/html5/thumbnails/31.jpg)
31
Current and Future WorkCurrent and Future Work
► Larger Site:Larger Site: Summer 2004 - Thulamela, South Summer 2004 - Thulamela, South
AfricaAfrica
► Personalized user experience Personalized user experience based on expertisebased on expertise
► Environment managementEnvironment management
![Page 32: Collaborative Mixed Reality Visualization of an Archaeological Excavation Hrvoje Benko Department of Computer Science Columbia University MIT talk – January](https://reader035.vdocuments.mx/reader035/viewer/2022070305/551477e0550346b2598b4622/html5/thumbnails/32.jpg)
32
AcknowledgmentsAcknowledgments► Special thanks to: Special thanks to:
Peter Allen, Alejandro Troccoli, and Ben Smith (CU Robotics Peter Allen, Alejandro Troccoli, and Ben Smith (CU Robotics Lab)Lab)
Ian Morris and Trinity Jackman (Stanford Archaeology Ian Morris and Trinity Jackman (Stanford Archaeology Center) Center)
Lynn Meskell and James Conlon (CU Dept. of Archaeology) Lynn Meskell and James Conlon (CU Dept. of Archaeology) Sajid Sadi and Avinanindra Utukuri (P5 glove) Sajid Sadi and Avinanindra Utukuri (P5 glove) Shezan BaigShezan Baig Mitsubishi Electric Research Labs (DiamondTouch table)Mitsubishi Electric Research Labs (DiamondTouch table) Alias Systems Alias Systems Microsoft Research Microsoft Research
► Funded in part by NSF ITR Grants IIS-0121239 and Funded in part by NSF ITR Grants IIS-0121239 and IIS-00-82961, and Office of Naval Research IIS-00-82961, and Office of Naval Research Contracts N00014-99-1- 0394, N00014-99-1-0683, Contracts N00014-99-1- 0394, N00014-99-1-0683, N00014-99-1-0249, and N00014-04-1-0005.N00014-99-1-0249, and N00014-04-1-0005.
![Page 33: Collaborative Mixed Reality Visualization of an Archaeological Excavation Hrvoje Benko Department of Computer Science Columbia University MIT talk – January](https://reader035.vdocuments.mx/reader035/viewer/2022070305/551477e0550346b2598b4622/html5/thumbnails/33.jpg)
Questions?Questions?
![Page 34: Collaborative Mixed Reality Visualization of an Archaeological Excavation Hrvoje Benko Department of Computer Science Columbia University MIT talk – January](https://reader035.vdocuments.mx/reader035/viewer/2022070305/551477e0550346b2598b4622/html5/thumbnails/34.jpg)
34