EXPLORING 3D VIRTUAL ENVIRONMENTS THROUGH OPTIMISED SPHERICALPANORAMA NAVIGATION

Andrea Ferracani, Lea Landucci, Pietro Pala

Universita` degli Studi di Firenze - MICCname.surname@unifi.it

ABSTRACT

Commonly, immersive and virtual reality systems simulatereal environments exploiting 3D computer graphics. Thisentails a considerable work to be done in models develop-ment and objects textures mapping in order to obtain a gooddegree of realism. Furthermore, the excessive complexityand the high rendering quality of the models can compro-mise system performance, especially in a web environment.This paper describes a vision based approach which allowsa user to immersively navigate a real cultural environmentthrough a lightweight web based system for the interactivewalk-through and browsing of an ordered sequence of spher-ical panoramas.

Index Terms Virtual reality, Sub-scene matching,Spherical images, Interaction design, Web application.

1. INTRODUCTION

In the last few years web browsers have been providingan increasing support to third dimension technologies, al-though 3D rendering engines are not fully ubiquitous andthere are still some issues regarding the performance and real-ism of these systems. Many researchers have then focused onpanoramic images due to their attractive and immersive dis-play usage, first of all focusing on how to build a panoramastarting from image sequences (two or more) [2], and then ad-dressing the panorama navigation issues. Several works ex-ist which use different mapping representations for outdoorpanorama navigation, for example Google Street View [1].However the navigation metaphor is often more intended togive users a good experience of exploring one panorama (pan-ning, zooming and titling), rather than to optimise the switch-ing from one panorama to another. Our goal is to enablesmooth transitions across panoramas so as to reduce their per-ceived discontinuity and to propose an innovative interactionmetaphor for a better multimedia fruition including 3D mod-els, PDFs, galleries of images/videos and indoor panoramasrelated to the Point Of Interests (POIs) distributed along a cul-tural walkthrough.The paper is organised as follows. We discuss the image-based method for panoramas transition in Section 2. Section 3

presents the web interface and its interaction design metaphor.Section 4 outlines conclusions and future work.

2. OPTIMISED SPHERICAL PANORAMANAVIGATION

A spherical panoramic image is created by warping the radi-ally undistorted perspective images onto a unit sphere assum-ing one virtual optical center in which the user is supposed towatch at the panorama. Navigation within a single panoramais provided by two main actions: dragging and zooming; nav-igation among different panoramas is performed through asmooth transition with the replacement of the panoramic tex-ture. In order to support the navigation between panoramasthe interaction design model uses a metaphor based on thezooming action of the user: the transition takes place onlywhen the current zoom level exceeds a particular threshold(see Sec. 2.1). In order to minimise the gap in the transitionbetween the two scenes, we studied and developed an algo-rithm that performs sub-scene matching.

2.1. Sub-scene matching algorithm

The sub-scene matching algorithm is exploited in order to findthe best zoom factor threshold for triggering the transition tothe next (or previous) panorama. Given two images Ik andIk+1 we want to find the rectangular sub-region of Ik whosecontent is most similar to Ik+1. The sub-region and Ik+1 musthave the same aspect ratio, although their size is expected tobe different. Let us introduce the following notation:

wk and hk are the width and height in pixels of the imageIk;

the function Crop(I, x,x, y,y) returns the rectangularsub-region of the image I identified by the bottom-left co-ordinates (x, y) and the top-right coordinates (x+ x, y+y);

the function D(In, Im) returns the dissimilarity betweenthe content of two images In and Im : zero in the case thatthe content of the two images is the same and a numbergreater than zero otherwise. The measure of dissimilarityis invariant to the size of the two images.

Identification of the rectangular sub-region of Ik that bestmatches the content of Ik+1 in correspondence to the inwarddirection is accomplished by minimisation of the followingcost function with respect to the three variables (x, y, ):

{(x0, y0, 0) = arg[minx,y,D(f(x, y, ), Ik+1)]

f(x, y, ) = Crop(Ik, x, x+ wk+1, y, y + hk+1)

(1)The result of Eq.(1) is the triplet (xo, yo, o) of the variablesthat minimise the cost function; they are the coordinates ofthe bottom-left vertex of the sub-region Po = (xo, yo) andthe zoom factor Zo = 1/o to be applied to the sub-regionof Ik in order to match the size of Ik+1. Experimentally,we observed that computation of the dissimilarity functionD(In, Im) through the distance of the image histograms pro-vides higher effectiveness compared to solutions based onscale invariant local keypoint descriptors such as SIFT. This ismainly caused by the fact that in the general case, the scenesrepresented in two consecutive panoramic images can differsignificantly in some parts, due to severe occlusions that cantake place depending on the 3D structure of the scene cap-tured by the panoramic images. To reduce the computationtime associated with the minimisation of the cost function andspeed up the computation of the histograms we adopted thetechnique of Integral Histograms [3].

3. APPLICATION INTERFACE

The system has been developed as a web application basedon HTML5, CSS3, Javascript and WebGL and has beenimplemented using the 3D open source library three.js1.The navigation among spherical images has been carried outthrough the dynamic replacement of the texture mapped intothe sphere, according to user interaction. The update of thetexture is done taking into account the optimal zoom factorthreshold for the transition (see Sec. 2.1). This can occurwhen the user zooms in the outward direction and exceedsthe threshold pre-computed by the sub-scene matching mod-ule. Otherwise, when the direction of zooming is not alignedto the outward direction the rendered image is progressivelymagnifiedand the field of view is reduced accordinglyuntil the maximum zoom level is reached. The applicationcan be configured in order to show interactive hotspots in thevirtual walkthrough. Each hotspot can be activated by usersand it provides several multimedia additional material aboutspecific POIs. Hotspots are constituted by four graphic iconsarranged in a circular menu. Each item represents one of thefollowing multimedia categories: 1) gallery of images/videos(yellow icon), 2) PDF (red icon), 3) 3D object (green icon) ,4) Indoor panorama (blue icon), cfr. Fig. 1. All these mul-timedia artefacts are shown inside a floating panel orientedcontextually to the panoramic environment and are arranged

1http://threejs.org/

Fig. 1. Examples of interaction with an hot-spot: Indoorpanorama

in real-time by the system with the best spatial position ac-cording to the user point-of-view. In this way, the appli-cation overcomes the limitation of standard panorama-basedinterfaces in which additional content (3D or 2D) is shownin two-dimensional lightboxes covering the main navigationarea. Furthermore, hotspots can be dragged using an handleat the centre in order to let users better organise the content inthe interface view. Context awareness is provided through amini-map at the bottom-left angle of the screen. Position andorientation are shown on the map and a little circular handleon the route path can be dragged in order to move to differentpanoramas. POIs, hotspots and associated multimedia mate-rials are searchable using an autosuggest input field.

4. CONCLUSION

In this paper we present a web tool for an immersive in-teractive walk-through in an urban cultural scenario and wepropose a new interface and interaction metaphor for access-ing cultural content. An optimisation method for the transi-tion between spherical images based on a Sub-scene matchingparadigm is proposed.

5. ACKNOWLEDGMENTS

The research was conducted as part of a project developedin collaboration with F.A.S.T S.R.L. (Fine Art Science andTechnology), which provided all the multimedia material, andfunded by Regione Toscana.

6. REFERENCES

[1] D. A. at Al. Google street view: Capturing the world atstreet level. Computer - IEEE, june 2010.

[2] O. Cogal, V. Popovic, and Y. Leblebici. Sphericalpanorama construction using multi sensor registrationpriors and its real-time hardware. In Multimedia (ISM),2013 IEEE International Symposium on, pages 171178,Dec 2013.

[3] F. Porikli. Integral histogram: a fast way to extract his-tograms in cartesian spaces. In Proceedings of IEEECVPR, 2005, 2005.