1
VIDENTE - 3D Visualization of Underground Infrastructure using Handheld Augmented Reality
Gerhard Schall1, Dieter Schmalstieg1
1 Institute of Computer Graphics and Vision, Graz University of Technology, Inffeldgasse 16a, 8010 Graz, Austria, {schall|[email protected]}, Tel: +43-(0)316-873-5011, Fax: -5050
Sebastian Junghanns2
2 GRINTEC GmbH, Anzengrubergasse 6, 8010 Graz, Austria, [email protected], Tel: +43-(0)316-383706-0, Fax: -20
Introduction This chapter outlines a research project called Vidente following the vision of registered 3D visualization of underground networks on handheld devices in real-time. Towards this aim, technology from interdisciplinary fields such as computer graphics, augmented reality, geographic information systems (GIS) and satellite navigation needs to be addressed. We highlight aspects of the Vidente system targeted on water systems operated by utilities from the water supply sector, which are already completely relying on their geo-databases for day-to-day operation of their assets. However, there is a noticeable gap between desktop GIS technology available in the office and access to this information in the field. To fill this gap, we propose to provide field workers with an intuitive three-dimensional visualization of the local underground network infrastructure using outdoor handheld augmented reality (AR) as depicted in Figure 1. The focus is on a next-generation mobile GIS system for utilities as well as telecommunication companies, supporting mobile workforces in the complete life cycle of water infrastructure, thus revolutionizing traditional planning, operation, maintenance, on-site inspection, fault management and decision-making. The project significantly advances mobile GIS in water engineering. Moreover, common field tasks concerning maintenance and operation are accomplished more efficiently while reducing unintended damage and increasing general safety on site. The system is intended to equip professionals, practitioners, water resources engineers, managers and decision makers working in water related arenas, utilities from the water sector, water boards and other government agencies with available information and advanced information technology tools to assist in on-site applications to geohydrological and environmental problems of urban waters.
2
Figure 1: View through the AR display deployed outdoors. In the image a second AR user with a handheld AR display can be seen. Underground infrastructure as well as wire frame building models are shown.
Current state of field information systems of utilities Mobile GIS extends Geographic Information Systems beyond the office to the field by incorporating technologies such as mobile devices, wireless communication and positioning systems. Mobile GIS enables accurate, real-time decisions, on-site capturing, storing, manipulating, analyzing and displaying of geospatial data. As an extension, the ability to interact with location based information was added to mobile GIS applications and services. Such services play an important role for on-site analysis, aiding critical decision making with information about underground infrastructure assets. Industrial mobile GIS can already be deployed in low end computing systems such as PDAs [ArcPad07]. Current commercial mobile GIS products include, for example, FieldWorker, GPSPilot, Fugawi, Todarmal, ESRI ArcPad, MapInfo, MapXmobile, Smallworld Field Information System or MapFrame FieldSmart. FieldWorker is used for exchanging information with mobile workers. GPSPilot and Fugawi are examples of traditional 2D maps intended for navigation. Todarmal provides the possibility to create map content (points, lines and 2D polygons) online in a layered manner. ESRI ArcPad is intended for managing point type GIS data, where digital photos can be attached to point information. The ArcPad comes with a support for routing with street map data. MapInfo MapXmobile is a development tool similar to ArcPad, intended for creating map applications. Smallworld Field Information System and MapFrame FieldSmart represent mobile GIS particularly aiming at needs of utilities, hence supporting a process-driven approach. Both applications support straightforward handling of extensive utility network datasets and efficient data reconciling with version-managed and longterm-transaction-based back-office GIS.
3
New methods for mobile GIS offered by VIDENTE Handheld augmented reality extends traditional 2D or 3D visualizations by overlaying registered visualizations over video footages. There is a clear trend away from bulky computer equipment towards handheld devices which are more lightweight and already socially accepted [Wagner03]. Over the last years only few ultra mobile PC installations have been used for handheld AR deployed especially in outdoor conditions, predominantly in urban areas [Schmalstieg05] [Schall07] [Veas08]. They replace laptop-based AR systems or backpack-based solutions [Feiner97] [Thomas98]. Although these systems can run simple visualizations, it is impossible to manage the large data sets needed for on-site monitoring using AR methods. In relation to the previous section, handheld devices exist that have been used in the exploration of GIS data. These include ARVINO, exploring viticulture data [King05], and Priestnall simple landscape visualization system [Priestnall06]. Examples of interaction tools with underground infrastructure using handheld AR are described in section “Visualization techniques” [Schall08a] [Schall08b] (see Figure 2 and Figure 3). Among others these tools comprise data retrieval capabilities, redlining functionality to annotate the geospatial assets and a virtual excavation tool to improve depth perception of complex underground infrastructure. Especially useful was the planning tool, which allows for visualizing projected assets superimposed over the real world. Moreover, the position of the projected asset can be changed on-site interactively.
Furthermore, Vidente can contribute to Horizontal Directional Drilling (HDD), which provides an efficient, economical system for installing underground lines and pipe without disturbing the surface environment. In such a task, a drill head with a miniature transmitter travels through the ground on a prescribed path, transmitting updated data to a computer on the drilling equipment. The operator monitors the input and adjusts the direction and movement of the drill head to avoid other underground infrastructure and efficiently reach the prescribed end location. After expanding the initial path, the cable is then pulled through the underground path providing a perfect path for water lines, but also power, telephone of fiber optic cables and lines. Underground Horizontal Directional Drilling is especially cost-effective compared to old-fashioned trenching, with its need for landscape repair and the additional problems of relocating
Figure 2. Vidente: handheld augmented reality device. Figure 3. Vidente: presentation of the subsurface infrastructure information at the client device (screenshot) (data courtesy of Salzburg AG).
4
and avoiding other underground systems. Underground Horizontal Directional Drilling can benefit from a technology like Vidente by allowing to visualize existing underground infrastructure registered in 3D with the real environment. Thus, giving visual assistance to the operator to support the monitoring process by X-Ray visualization views beneath the ground. An even more challenging but promising goal is to replace true surveying tasks with AR, which up to now required the use of specialized equipment such as a tachymeter. For example, users wishing to document the path of a projected pipe after on-site inspection of the best route previously had to record a sequence of waypoints using conventional surveying, and then import these waypoints into the geo-database in the planning office to create a digital asset. We suggest using 3D interaction techniques to allow workers to create new digital geo-referenced assets. For example, a worker can survey a series of waypoints by intersecting the ray from a “virtual laser pointer” with the digital elevation model. Accuracy concerns notwithstanding, it may be an order of magnitude faster to record several such waypoints from one location rather than having to physically follow the path of the planned pipe. However, the creation of some objects may require triangulation from at least two physical locations for sufficiently precise input, as suggested by [Höllerer01] and [Piekarski01]. Explanation of change in process Technological developments in imaging and vision technology, geoinformatics, computer graphics and augmented reality (AR) are promising more and more capabilities not only in visualizing the real world but also in spatial data acquisition and management. Recent years showed a strong trend towards handheld devices such as (ruggedized) Ultra Mobile PCs, PDAs or even smart phones, since these devices are already equipped with sensors necessary for AR. Furthermore, mobile GIS extends geographic information systems from the office to the field by incorporating the before mentioned technologies. Mobile GIS enables workers for on-site capturing, storing, manipulating, analysing and displaying of geospatial data and therefore provides clear benefits for mobile workforces. Current trends of mobile field systems are dominated by quality and completeness control in the field, seamless dataflow between office and field and vice versa, no redundancies in data and workflows. With the approach of providing the entire geo-database in the field, mobile workforces can react on changing conditions too. Moreover, for improved efficiency, paper plans are increasingly replaced by notebook computers taken to the field to directly consult the GIS. Just like the paper plan, the GIS uses two-dimensional models to represent the geographic data. There is also a certain trend observable towards 3D GIS trying to eliminate shortcomings resulting from two-dimensional representations of objects. 3D GIS databases contain 3D data structure representing both the geometry and topology of the 3D shapes, and allowing 3D spatial analysis. A variety of industries such as government, utility/infrastructure and public safety, are performing a paradigm shift towards Smart Grids. “Smart Grids”, i.e. intelligent, self-optimizing utility networks [Technology Review08] require improved tools for utility workers in the field. This can also be inferred from the trends in GIS research mentioned by geoscientist Huisman
5
[Huisman08]: • The increasing availability and use of shared derivative data artifacts; • The increasing demand for temporal and dynamic functionality in geo-information; • The increasing seeking for representations of objects true to their nature.
This raises various issues with regard to the deployment of geospatial data. Specifically, the last trend highlights the importance of AR for providing realistic 3D visualizations for mobile GIS applications. AR has the potential to remove the need for a mental transformation from map to reality. As the supply and utility grids get smarter, the need for smarter mobile systems grows. Mobile field information systems for supporting the mobile workforce in on-site tasks are increasingly vital. Expected impact As field operation is labor intensive and hence costly, improved technology for this area can lead to savings and improved productivity. Vidente has the potential to do away with conventional maps (printed or digital), which are often difficult to interpret, to support mobile workforces with registered three-dimensional visualizations of underground assets during the whole life cycle of underground water assets. Trenching and excavation is recognized as one of the most hazardous construction operations. Pre-planning is vital to accident-free trenching; safety cannot be improvised as work progresses. Vidente can contribute to reduce the exposure of field workers in the utility management sector by reducing the number and duration of excavation works of any kind, and subsequently to better pre-planning and precise location of the operation to be performed. Secondly, Vidente can contribute to safety by reducing the risk of accidental pipeline rupture or leakage and consequent exposure of workers to harmful gases, explosive gas atmosphere, thanks to a comprehensive knowledge of buried infrastructures. From an environmental point of view, Vidente provides the necessary research which leads to a fundamental platform for a 3rd generation field information system (1st generation: analog paper maps and plans, 2nd generation: digital maps on laptop computers of PDAs). Using high-precision tracking of the mobile field worker this system aims to allow executing “key-hole surgery” on the underground infrastructure operated by enterprises of the utility sector. Such a development is expected to require less excavations; limited to the necessary, and thus offering faster execution of field work, reduced interruption to traffic and consequently less impact on the environment. VIDENTE system As a basis for the system, we use handheld AR technology for visualizing registered 3D underground models in real-time. The three-dimensional geometry must be extracted from a conventional database system and interpreted on the fly as a 3D visualization. However, the three-dimensional models are not stored persistently. Rather, the underlying data is stored
6
persistently and managed in the utilities’ geospatial databases. Hence, we always access the most current data version and can benefit from all the advantages of a powerful database system such as data versioning, loss prevention, recovery, integrity enforcement and comfortable operations for retrieval, insertion and update. Data redundancy and inconsistency among spatially overlapping models are eliminated since all models refer to a common data source. A lean and generic GML application schema (VidenteGML) serves to encode the underlying geo-referenced utility asset data issued from the data server [Junghanns09]. Up to now the pipeline is run as a semi-automated offline process. That is, the area and the objects of interest are interactively selected by the user, exported and then uploaded to the client for conversion. Future research work of our group aims at an increased degree of automation of this process.
Figure 4: Vidente data pipeline – envisioned data flow.
Vidente is based on a multi-tier system architecture with a mobile front-end and an operational geospatial database as a back-end (Figure 4). The mobile front-end is a handheld client device, which is designed as video see-through. Hence, scenes are assembled at the client device in real-time by merging continuously streamed video footage with geo-referenced computer graphics considering the client’s currently tracked position and orientation. The Vidente system is build upon the Studierstube AR Framework ensuring the rendered AR scenes are adjusted continuously as the user moves around [Schmalstieg02]. Registration in 3D requires the capability to perform accurate global localization and pose tracking in real-time. We have equipped a handheld setup with tracking sensors designed for outdoor use.
Handheld AR platform The handheld AR device (Figure 2) is built around an Ultra Mobile PC (UMPC, Sony® VAIO® UX1) mounted on a special handheld frame equipped with joystick-like controls for user interaction (Figure 1, cf. [Kruijff07]). Employing a UMPC as the core unit of our setup allows for a completely self-contained system providing sufficient computing power for three-dimensional graphics and several hours of battery-powered operation in the field. Those are crucial prerequisites for the device’s intended outdoor use in unprepared locations. Furthermore, the client platform depicted in Figure 2 comprises a GPS module (NovAtel® OEMV1® receiver) and an inertial measurement unit (XSens® MTi-G™) for respective position and orientation tracking, a highly compact video camera (Ueye® 2210) for streaming video footage of the immediate environs and a UMTS adapter for wireless data exchange.
7
The advanced ergonomic form factor of the handheld AR device enables users to hold the device for extended periods of time. Particular attention was paid to an ergonomic shape of the two-handed carrier frame permitting acceptable weight and grip of the device and easy manipulation of the joystick-like controls using thumbs and index fingers (cf. [Kruijff07]). Interaction with the application running on the device is solely effected by means of the afore-mentioned controls. Tedious and error-prone touch screen inputs using a stylus are thus avoided. Outdoor tracking using sensor fusion Apart from the platform technology, the main technological challenge to general mobile Augmented Reality is tracking and registration. AR requires extremely accurate position and orientation tracking to align, or register virtual object with the physical objects. The current tracking solution for outdoor use relies on GPS and inertial measurement unit (IMU). The two sensor units are fused using an extended Kalman filter to yield a continuous pose (6-degree-of-freedom position and orientation) at a rate suitable for our real-time visualization system (20-25Hz). Currently a GPS receiver using EGNOS correction signals is used, which allows positional accuracies up to the m-level. As part of our ongoing research we work on the integration of terrestrial correction data services (DGPS / RTK-GPS) as well as on the integration of vision-based tracking approaches (cf. [Reitmayr07]) to leverage the required accuracy. Modeling of 3D underground infrastructure through transcoding For AR purposes, creating 3D models of large environments is a research challenge. Automatic methods, semiautomatic or manual techniques can be employed. Among them are 3D reconstruction from satellite imagery, 3D imaging with laser range finders and procedural modeling techniques. For the generation of accurate models of the underground infrastructure networks other techniques are needed such as transcoding (see Figure 5 and Figure 6) [Schall08c]. Three-dimensional representation and visualization of urban environments are employed in an increasing number of applications, such as urban planning and emergency tasks. A procedure to make use of large productive geospatial databases is called transcoding: a process of turning raw geospatial data, which are mostly 2D, into 3D models suitable for standard rendering engines. Users from companies like in the utility sector expect reliable data representations, so strict dependence on real-world measurements is necessary. Consequently, we generate the models from data exported from geospatial databases in the standard Geography Markup Language1, (GML). There are derivatives of GML, such as CityGML [Kolbe05], which is a specialization of the GML language for 3D visualization 3D city models requiring a special browser. Instead, we use a standard scene-graph structure which enables to preserve the semantic data from the geo-database in the resulting 3D models. This has the advantage that semantic information can be used to change the appearance of the 3D model in real-time. In [Mendez08] the visualization techniques are described in more detail. There has been other work on forwarding database
1 www.opengeospatial.org
8
information to scene-graphs with a database, for example X-VRML [Walczak03], but these types of approaches generally do not involve on-the-fly procedural modeling. The focus of existing research is mainly targeted on above surface city models. Notable exceptions include the VISTA project [Beck07], which tries to obtain 3D underground models in the UK and use automated recognition for some of the modeling tasks. However, these projects do not aim at models that are suitable for mobile AR. One of the first projects dealing with AR for subsurface infrastructure is [Roberts02].
Figure 5: Transcoding pipeline from Smallworld™ geo-database 3D models of underground infrastructure and extruded footprints.
Figure 6: Combining modeling techniques. This image is a composite model of a procedural created T-junction with valve models from a stock library. By using subdivision surfaces complex junction shapes can be created (Left). The geospatial 3D model of the underground infrastructure is superimposed on a construction site (Right). Tools/Visualization techniques To optimize the benefit of an AR application, all presented information has to be designed towards an intuitively understandable visualization. However, the simultaneous representation of
Extruded FootprintsSmallworld™ GML Output Open Inventor Format
9
both virtual and real information introduces a number of difficulties. For example, virtual data always overrides real world information, which is especially problematic when presenting subsurface structures in so-called X-Ray visualizations. Therefore, virtual and real information have to be carefully chosen to avoid problems of depth perception, caused by a loss of information. To handle this problem, work has been shown focusing on the modifications of hidden structure, while other research was concentrating on the stylization of the occluding objects. For example, Livingston et al. [Livingston07] discuss drawing styles of hidden structure with the goal to support their spatial perception, and Bane et al. [Bane04] presents interactive tools to select certain parts of hidden geometry. In the following, we describe a series of tools we have implemented for visualization and interaction purposes with underground infrastructure models. Filtering tool Desktop GIS systems offer advanced possibilities for filtering and selecting information to avoid cluttering. Such detailed attribute selection tends to be too complicated for interaction in handheld AR. Instead, as shown in Figure 7, we let the user select a region of interest first, and then turn on 3D features based on pre-grouping into asset categories (gas, water, buildings and so on). This two-step filtering approach reduces clutter to a manageable amount with only a minimum of interaction. Figure 8 depicts and close-up view of a selected underground pipe network.
Figure 7: X-Ray view along the street showing water mains including several valves in blue color. (data courtesy of Graz AG - Stadtwerke fuer kommunale Dienste)
Figure 8: Close-up inspection of an underground asset (screenshot) (data courtesy of WVV)
Excavation tool Indiscriminately overlaying hidden information on top of visible real-world entities introduces depth perception problems. Virtual objects appear to float on top of the real ones because of overdraw. Therefore we employ an excavation tool resembling a hole in the ground, thereby providing plausible interpretation of depth through partial object occlusion as well as motion parallax. (see Figure 9). The excavation tool is implemented using a magic lens technique, filtering the content based on contextual information [Kalkofen07] derived from the attribute
10
data in the GIS. The lens is initially positioned in front of the user, but can be adjusted using controls on the AR device.
Figure 9: Excavation tool. Screenshots demonstrating improved depth perception by adding specific depth cues (data courtesy of Graz AG - Stadtwerke fuer kommunale Dienste).
Snapshot tool For documentation, field workers like to freeze an image at any point in time and take a snapshot, to be analyzed later in the planning office. A dedicated button on the AR device triggers such a snapshot (see Figure 10).
Figure 10: Snapshot of the augmented live video.
Metadata Querying tool We have also implemented a metadata querying tool, which helps the user to visualize the meta-information of the infrastructure, such as part number, ownership etc. This meta-information is obtained from the original geospatial data and stored as non-geometrical attributes on the 3D model. As depicted in Figure 11, a crosshair target can be positioned on top of an asset, revealing associated meta-information.
11
Figure 11: Metadata Querying tool. Using a cross-hair a user can select the water line and query related semantic information (screenshot) (data courtesy of Graz AG - Stadtwerke fuer kommunale Dienste)
Figure 12: Redlining tool. Cubes are placed alongside the water lines and a circular area is marked in green color. (data courtesy of Graz AG - Stadtwerke fuer kommunale Dienste)
The above mentioned tools can be comprised as an inspection toolset useful for visualization and inspection purposes. Moreover, we have implemented an interaction toolset providing mobile workforces with more interactive capabilities. Annotating the geospatial model A redlining tool in an augmented reality style provides field workers with on-site redlining capabilities. The redlining tool enables the outdoor user to annotate and interact with geospatial objects. The user can choose a symbol from a predefined palette of symbols (e.g., damage, safety area, or maintenance area), that can be placed in the geospatial model. Using the point-and-shoot metaphor of the AR device, the user can place the selected symbol at the point of intersection with the geospatial model. Furthermore, the tool enables the user to mark areas on the terrain. The radius of the area to be marked can be changed by varying the pitch and yaw of the device (see Figure 12).
Surveying locations in the geospatial model Besides placing an annotation to a location in the geospatial 3D model the user also has the possibility to survey locations by intersecting the current position with the geospatial model (underground infrastructure, DTM or buildings). This enables a field worker for example to survey a single spot or the location of a trench. Small cubes indicate the surveyed locations by the field worker (see Figure 12)
Interactive validation of object placement Some applications demand to inspect, validate or modify the placement of specific structures in the environment. This can be necessary if either the GIS is known to be incomplete, so that planning exclusively in the office is not feasible, or if plans from contractors are obtained without geo-referencing. In this case, surveying in the field and planning the actual location of
12
the asset can be integrated in one interactive feature of the AR system, assuming that a 3D representation of the inspected structure is already available.
Figure 13: Noise protection barrier to be erected alongside a railroad track. The planned barrier is subject to the on-site in-
spection with the AR system to determine overlapping areas with existing underground infrastructure (data courtesy of ÖBB).
For example, Figure 13 shows a noise protection barrier to be erected alongside a railroad track. The barrier has been planned by a contractor, while the exact placement of the barrier is subject to the on-site inspection with the AR system. The barrier must not be built on top of existing underground utility infrastructure, to assure that maintenance of the utilities is not affected. In order to do that, the field workers determine various possible placements in an on-site planning discussion.
Visualization of abstract information Furthermore, Vidente can provide a verification toolset for visualizing abstract data. Thus, allowing for quality control and verification of underground assets. Visualization of legally binding land-register data is an important task for utility suppliers, since this information is usually difficult to find on-site. A wide range of abstract information, such as parcel borders, parcel areas, ownership and servitude rights are relevant for this task. Figure 14 and Figure 15 compare an exocentric view of the map and an egocentric AR view of parcels.
Figure 14. Exocentric view at land register and underground services data as available in conventional two-dimensional GIS visualisations (Graz Geodatenserver).
Figure 15. Egocentric view at land register (dark grey on the right highlighting extents of adjacent parcel) and underground services data using AR techniques.
13
Deployment examples The development of Vidente has been driven by a continuous close collaboration with potential end users from utility companies. A number of workshops with technical field personnel have revealed the high potential of mobile AR for common field tasks performed as a part of such technical business processes as maintenance, outage management, inspection, as-built record keeping, decision support, network planning and network construction. In practical terms the following use cases, which are not limited to underground water infrastructure, were identified:
• Trench inspection: Underground pipe networks require regular inspection to locate leaks at
an early stage. Vidente delivers a location-dependent three-dimensional visualization of the hidden network as the inspection worker follows down the trench or does inspection in the immediate trench environs. This improves remarkably the worker’s orientation in the field and his understanding of the current complex underground layout. Furthermore, inspection workers can record spots of leaking by simply dropping a new redlining geometry in the Vidente application.
• Preparation of digging activities: A common task in the preparation of a construction site
comprises spray-marking the layout of the buried assets onto the pavement. Using Vidente the field worker sees a virtual overlay of the buried assets to spray-mark. The overlay serves as a virtual template for placing the spray-marks at the correct spot. There is no more need for tape measurements and error-prone map interpretation.
• Instruction of contractors: Contractors are instructed by utility staff members before any
digging activities take place. Vidente provides a more intuitive visualization of the underground network of the spot of the projected construction site compared to what dense two-dimensional paper map drawings could convey. Contractors see the as-built situation as it is and do not have to transform this information from a two-dimensional map into the real world.
• Visual guidance for digging: Digging in the vicinity of safety-critical assets or along legal
boundaries (e.g. parcel boundaries) requires a high degree of vigilance of the machine operator. Being mounted on the machine’s dashboard, the Vidente client provides a real-time visualization of the trench line the operator is supposed to dig along. Abstract information such as parcel boundaries is made visible, as well (see Figure 14 and Figure 15). Highly critical assets may be highlighted using specific colour coding to draw attention to them.
• Locating of buried damaged cables: It is a common issue that low voltage cables are
slightly damaged while being run into the trench. The trench is covered and the cable still functions for a certain period of time before it fails. Locating such spots of interruption is a tedious task. Vidente supports field workers by displaying an overlay of the trench to be inspected and the assets possibly concerned. Using the Vidente application field workers can virtually select the concerned asset, query technical data related to it and mark it as affected.
14
• On-site verification of projected assets: Utility companies employ back-office tools such as their network information and enterprise resource planning systems for planning extensions to their existing networks. In this way the less cost-intensive option for say a new trench layout is computed. However, for most European utilities it is obligatory to verify the projected assets in situ to reveal conflicts, which may not have been identified by the back-office systems due to limited data quality. Hence, Vidente helps to take the planning out to the field and to visualise the projected assets by virtually placing them into their future environs. Thus conflicts are easily identified.
• Assistance for operation and maintenance works: One of the strong points of AR is the
capability to enhance real-world vision by abstract information and to associate that information visually with a physical feature in the real world. Say, a field worker is examining a particular asset while performing maintenance work. Looking at the asset using Vidente, he is provided with concise technical or task-related information by means of floating labels. Geo-referenced colour-coded symbol overlays indicate spots to be examined more closely. Alternatively, the field worker can select the virtual representation of the asset and explicitly query the related technical information.
• On-site correction of legacy data: Most utilities still struggle with the positional accuracy
of as-built asset data migrated into their geospatial databases from legacy datasets, paper maps or even hand drawings. One approach to overcome this issue is to take the data to the field and correct it in situ corresponding to the real-world situation. Using Vidente the legacy asset data can be viewed in the field as a virtual overlay superimposed onto the view of the real-world situation. By interactively modifying the displayed object geometries, they can make the virtual data match the real-world situation. The thus modified data is reconciled with the geospatial database.
The above use cases demonstrate that Vidente have the potential to significantly contribute to an improved day-to-day productivity of utility field forces from the water supply sector. However, Vidente is not designed to fully replace existing solutions of mobile asset management, but rather to supplement them with an additional mode of information providing. It depends on the work task if a two-dimensional exocentric small-scale view at the data as provided by conventional mobile GIS is preferable, or if a three-dimensional more plastic data visualization from an egocentric perspective proves useful. Vidente clearly aims at close-up investigations of assets and at inspections in the immediate environs of the field worker’s current position. As such we consider it to be integrated with mobile GIS applications to provide users with the opportunity to switch between two different modes of data presentation. Enabling an automated roundtrip of the data flow from the office to the field and back to the office is a key issue of our on-going research work. Summary and Discussion In this chapter, we have presented Vidente, a location- and context-aware handheld AR system for on-site visualization and interaction, which addresses the workflow optimisation of common field tasks with utilities. By means of a more intuitive way of information conveying, remarkable
15
time saving can be achieved employing such a system. Moreover, potential fields of application were outlined and discussed. A first fully functional prototype of the handheld AR system Vidente is available and provides a set of tools for direct user interaction with the presented information on buried utility assets. The robustness, attractiveness, and efficiency of technology are essential for a sucessful application of spatial AR in outdoor environments. Vidente has the potential for increasing the efficiency of mobile workforces during the complete life cycle of water infrastructure, thus revolutionizing traditional planning, operation, maintenance, on-site inspection, fault management and decision-making methodologies. A brief overview of the technology was given in this chapter that tells the first sucess stories and provides a promising perspective for upcomming innovations. The technology can not only be used for visualization of geospatial content but, for example, also of environmental data from sensor networks as outlined in Hydrosys in section 3 describing GIS Applications in Hydrology. Among others, further improvements in our ongoing research will focus on advanced global tracking using DGPS / RTK-GPS employing terrestrial correction data services and digital processing techniques while keeping an ergonomically acceptable form factor. Acknowledgments This work was funded by the Austrian Research Promotion Agency (FFG) under contract no. BRIDGE 811000, FIT-IT 820922 and the European Union under contract no. FP6-2004-IST-427571, and the Austrian Science Fund (FWF) under contract no. Y193 and W1209-N15. Moreover, we would like to express our gratitude to the following organisations for providing both real-world test datasets and valuable input to our research activities from the end-user perspective: Salzburg AG für Energie, Verkehr und Telekommunikation, Würzburger Versorgungs- und Verkehrs-GmbH (WVV), ÖBB-Infrastruktur Bau AG, Elektrizitätswerk Gösting V. Franz GmbH, Energie AG Oberösterreich, Wienstrom GmbH, Graz AG - Stadtwerke fuer kommunale Dienste, Stadtvermessungsamt Graz and Innsbrucker Kommunalbetriebe AG. References
[ArcPad07] ArcPad, ArcPad - Mobile GIS Software for Field Mapping Applications, available at: http://www.esri.com/software/arcgis/arcpad/index.html. 2007. [Beck07] Beck, A R; Fu, G; Cohn, A G; Bennett, B; Stell, J G, A framework for utility data integration in the UK. In: Coors, V., Rumor, M., Fendel, E. M. & Zlatanova, S. (Eds) Urban and Regional Data Management - Proceedings of the Urban Data Management Society Symposium 2007. [Feiner97] Feiner, S., et al. A Touring Machine: Prototyping 3D Mobile Augmented Reality Systems for exploring the Urban Environment. In: Proceedings of ISWC'97. 1997. [Höllerer01] Höllerer, T., Feiner, S., Hallaway, D., Bell, B., Lanzagorta, M., Brown, D., Julier, S., Baillot, Y., Rosenblum, L. User Interface Management Techniques for Collaborative Mobile Augmented Reality. In Computers and Graphics 25, 2001, pp. 799-810.
16
[Huisman08] Huisman O. and Pip Forer, “Progress in the application of Time-Geographic concepts to urban micro-process”, GISCIENCE 2008. [Junghanns09] Junghanns S., Schall G., Schmalstieg D., "Employing location-aware handheld augmented reality to assist utilities field personnel", To appear in Journal of Location Based Services archive, 2009. [Kalkofen07] Kalkofen D., Erick Mendez, Dieter Schmalstieg: Interactive Focus and Context Visualization in Augmented Reality. Proc. 6th IEEE International Symposium on Mixed and Augmented Reality (ISMAR'07), pp. 191-200, 2007. [King05] King, G., W. Piekarski, and B. Thomas. ARVino - outdoor augmented reality visualization of viticulture GIS data. In: International Symposium on Mixed and Augmented Reality 2005 (ISMAR'05). 2005. [Kolbe05] Kolbe T., Groeger G., Pluemer L, “CityGML – Interoperable Access to 3D City Models,” Proceedings of the Int. Symposium on Geoinformation for Disaster Management,” Springer, 2005. [Kruijff07] Kruijff, E. and Veas, E., Vesp’R – Transforming Handheld Augmented Reality.Proceedings of the 6th IEEE International Symposium on Mixed and Augmented Reality (ISMAR'07), 13–16 November 2007 Nara. IEEE, 277–278. [Mendez08] Mendez E., Schall G., Havemann S., Junghanns S., Schmalstieg D., “Generating 3D Models of Subsurface Infrastructure through Transcoding of Geo-Databases” In: IEEE Computer Graphics and Applications (CG&A), Special Issue on Procedural Methods for Urban Modelling, no. 3, May/June 2008. [Piekarski01] Piekarski Wayne, Thomas Bruce: Tinmith-Metro: New Outdoor Techniques for Creating City Models with an Augmented Reality Wearable Computer. In: 5th Int'l Symposium on Wearable Computers, pp 31-38, Zurich, Switzerland, Oct 2001. [Priestnall06] Priestnall, G. and G. Polmear. Landscape Visualization: From lab to field. In: First International Workshop on Mobile Geospatial Augmented Reality. 2006. [Reitmayr07] Reitmayr, G. and Drummond, T., Initialisation for Visual Tracking in Urban Environments. Proceedings of the 6th IEEE International Symposium on Mixed and Augmented Reality (ISMAR'07), 13–16 November 2007 Nara. IEEE. [Roberts02] Roberts, G.W., Evans, A.J., Dodson, A.H., Denby, B., Cooper, S.J. and Hollands, R.J. The use of augmented reality, GPS and INS for subsurface data visualization. In Proc. of XXII International Congress of the FIT, 2002, Washington DC, USA. [Schall07] Schall, G., et al. Handheld Geospatial Augmented Reality Using Urban 3D Models. In: Proceedings of the Workshop on Mobile Spatial Interaction, ACM International Conference
17
on Human Factors in Computing Systems (CHI´07). 2007. [Schall08a] Schall G., Mendez E., Kruijff E., Veas E., Junghanns S., Reitinger B., Schmalstieg D., “Handheld Augmented Reality for Underground Infrastructure Visualization” In: Personal and Ubiquitous Computing, Special Issue on Mobile Spatial Interaction, Springer, 2008. [Schall08b] Schall, G., Mendez, E. and Schmalstieg, D., Virtual Redlining for Civil Engineering. Proceedings of the 7th IEEE International Symposium on Mixed and Augmented Reality (ISMAR’08), 15–18 September 2008 Cambridge. IEEE. [Schall08c] Schall, G., Junghanns, S. and Schmalstieg, D., The Transcoding Pipeline: Automatic Generation of 3D Models from Geospatial Data Sources. Proceedings of the 1st International Workshop on Trends in Pervasive and Ubiquitous Geotechnology and Geoinformation (TIPUGG 2008), 23 September 2008 Park City. [Schmalstieg02] D. Schmalstieg, A. Fuhrmann, G. Hesina, Zs. Szalavari, L. M. Encarnação, M. Gervautz, W. Purgathofer: The Studierstube Augmented Reality Project PRESENCE - Teleoperators and Virtual Environments, Vol. 11, No. 1, MIT Press [Schmalstieg05] Schmalstieg, D. and G. Reitmayr. The World as a User Interface: Augmented Realityfor Ubiquitous Computing. In: Proceedings of the Eurographics Central European Multimedia and Virtual Reality Conference. 2005. [Schmalstieg07] Schmalstieg, D., Reitmayr, G., Schall, G., Newman, J., Wagner, D., Ledermann, F., Barakonyi, I. Managing Complex Augmented Reality Models. In IEEE CG&A, Special Issue on 3D Documents, no. 4, July/August 2007. [Technology Review08] “The smart grid”, Technology Review, Sept/Oct 2008. [Thomas98] Thomas, B., et al. A Wearable Computer System for Augmented Reality to support Terrestrial Navigation. In: Proceedings of ISWC'98. 1998. [Veas08] Eduardo Veas, Ernst Kruijff: Vesp'R: design and evaluation of a handheld AR device. IEEE/ACM international Symposium on Mixed and Augmented Reality 2008, Cambridge. [Wagner03] Wagner, D., Schmalstieg, D. First Steps Towards Handheld Augmented Reality. In: Proceedings of the 7th International Conference on Wearable Computers (ISWC ´03), 2003. [Walczak03] Walczak K., W. Cellary: X-VRML for Advanced Virtual Reality Applications. IEEE Computer, 89-92, March 2003.
