virtual reality and simulation...

VIRTUAL REALITY AND SIMULATION (2B)

AR: AN APPLICATION FOR INTERIOR DESIGN 115TOAN PHAN VIET, CHOO SEUNG YEON, WOO SEUNG HAK, CHOI AHRINA

GREEN CITY 125P.G. SHIVSHANKAR, R. BALACHANDAR

RETRIEVING LOST SPACE WITH TANGIBLE AUGMENTEDREALITY 135IRENE RUI CHEN, MARC AUREL SCHNABEL

SOME PHENOMENA IN THE SPATIAL REPRESENTATION OFVIRTUAL REALITY 143YEN-LIANG WU

CONQUERING NEW WORLDS 153NING GU, LEMAN FIGEN GUL, ANTHONY WILLIAMS, WALAIPORN NAKAPAN

COMPUTER GAME MODDING FOR ARCHITECTURE 163RUSSELL LOWE

MODULATIONS OF VOXEL SURFACES THROUGH EMOTIONALEXPRESSIONS TO GENERATE A FEEDBACK LOOP BETWEENPRIVATE MOOD AND PUBLIC IMAGE 173MATTHIAS HANK HAEUSLER

AR: AN APPLICATION FOR INTERIOR DESIGN

TOAN PHAN VIET, CHOO SEUNG YEON, WOO SEUNG HAK,CHOI AHRINADigital Architectural Design Lab, School of Architecture & Civilengineering, Kyungpook National University, South [email protected], [email protected], [email protected],[email protected]

Abstract. This paper presents an AR application (Augmented Reality)for interior design. Using AR for interior design, virtual furniture can beintegrated into the real world for improved designer-client collaborationand visualization. In our experiment, an user can see several virtualfurniture are overlaid on and they can communicate with 3D virtualfurniture data on user interface dynamically and flexibly, furthermoreproperties of virtual furniture could be adjusted through Occlusion basedinteraction methods for Tangible AR, etc.

Keywords. Augmented reality: ARToolKit; interactive augmented reality;tangible augmented reality; interior design.

1. Introduction

We all know that how difficult it is the right table or any furniture at furniturestore before deciding to buy, let alone taking it out and trying place it in yourroom. Hence, Augmented Reality (AR) technology has been proposed forinterior design applications by several authors, for example Koller; etc (1997).The related devices typically include data glasses hooked to a portable PC(HMD). A light weight solution is to use a PDA, example as proposed by theAugmented Reality Team in Finland (2003). However, these devices are notcommonly available for home consumers.

This paper presents augmented reality (AR) system architecture for design/sale/ordering of interior design by overlaying virtual furniture onto the physicalenvironment by an ordinary PC system. A tracking marker is placed on thefloor to define the scale and coordinate system of the room. Next, the software

116 T. P. VIET, C. S. YEON, W. S. HAK, C. AHRINA

system allows to choice of virtual furniture on screen to be placed into room.In the AR scene, the 3D virtual furniture integrates in real environment andstands beside several real furniture. Our experiment is implemented using justbasic home equipment; involve PC, webcam and printer. We intend it to besimple enough to operate for home users.

2. Augmented Reality Technology

2.1 AUGMENTED REALITY TECHNOLOGY

The Augmented Reality (AR) is new technology that involves the overlay ofcomputer graphics on the real world. In AR, the user can see the real worldaugmented with virtual objects and can interact with virtual objects. The AR iswithin a more general context termed Mixed Reality (MR), which refers to amulti-axis spectrum of areas that cover Virtual Reality (VR), Augmented Reality(AR), telepresence, and other related technology (Figure 1).

Figure 1. Simplified representation of Paul Milgram’s Reality- Virtual continuum

Augmented Reality systems combine digital information and real world ina way that a user experiences this as whole. An important property is especiallythat virtual objects are located to the right place and position. The TrackingSystem is one of the most important problems in AR system need to solve. ARsystem follows dynamically the user’s point of view and keeps virtual objectsaligned with real world objects. The basic components in AR applications area display, camera and a computer with application software. Various differentkinds of hardware can be used for this, for example camera phones, PDAs, lap-tops, HMDs, Wearable computer system etc.

Typically, a camera is attached to the display device, which shows the realtime camera view through its screen. To determine the relation between realand virtual world, the HIT LabNZ’s ARToolKit library was selected. ARToolKituses computer vision techniques position and orientation of the real cameraviewpoint relative to a real world marker. Next, ARToolKit defines and calculatethe position of virtual coordinates. Since the virtual and real camera coordinatesare matched, the computer graphics are drawn precisely as an overlay on top ofthe real-world marker (marker card).

AR: AN APPLICATION FOR INTERIOR DESIGN 117

2.2 AUGMENTED REALITY TECHNOLOGY IN ARCHITECTURE

Recently, AR technology is being considered as a new method of designapproach to architecture. A lot of AR experimentations and researches aredirected toward the architectural design process as strong tool. In belowexperiment, the 3D virtual house appears with real size in real world andeveryone with AR handheld devices can walks around and through it (Figure2, left). Another AR application is aimed at civil engineering as a highperformance user interface. An system shows users a supply pipe system, andadditional components in a real-world space (Figure 2, right).

Figure 2. Virtual building in AR scene and virtual supply pipe combined with real engine.

3. Interior Design In A Digital Environment

3.1 PROPERTIES OF INTERIOR ARCHITECTURE DESIGN

Typically an interior designer works with predetermined space, a designerdefines space to give it meaning, achieves harmony and lend it aura. There arethree basis principles of interior design-color, scale and proportion. For thisreason, our AR system is built to focus on how user design and control can bedynamic and flexible while addressing the three basis principles above. Usingour AR system, the user can adjust properties of virtual furniture by directinteraction, and can arrange all virtual furniture directly while in the real-worldenvironment.

3.2 SYSTEM DESIGN

For implementation, two separate modules were developed; one for creatingand management the 3D database, and the other one for displaying, Figure 3.


Figure 3. System Diagram

After CAD geometry of furniture is loaded from a database, the positionand direction of views for user is calculated based on data marker tracking.Simultaneously, the location-and direction based on geometry data viatransformation matrices produces an image that aligns with other objects inthe real-world view. Figure 4 summaries the tracking and display process ofthis system. Furniture graphic data is saved in the database, generated by aCAD application, e.g. 3DSMax software. OpenGL renders the final graphics.For tracking the physical object, we use a vision based tracking method. TheARToolKit software library is especially used for calculating the 3D positionand orientation of virtual marker.

Figure 4. The augmented reality tracking and display process: the computer-generatedgraphical augmentation is integrated into the user’s view of real world

3.3 SOFTWARE

The CAD applications handle management of the building geometry data andlink it to the database. Next, the AR software retrieves and displays data on the


position and orientation in defined environment. 3DSMax provides supportfor the next customization required for ARToolKit. 3DSMax produces a VRMLfile of model which has type *.wrl extension (Figure 5). One of the keydifficulties in developing AR applications is the problem of tracking the user’sviewpoint. In order to know from what viewpoint to draw the virtual imageryalign with real world objects, the AR application needs to know where the useris looking at the real world. The HIT LabNZ’s ARToolKit software usescomputer vision algorithms to solve this problem, the video tracking librariescalculate the virtual camera position and orientation relative physical markersin real time.

Figure 5. Generate *.wrl file

3.4 HARDWARE

Figure 6. Marker pattern and mk_patt.exe capture image for generating marker process

In our experiment, AR devices are based on ordinary PCs that run WindowXP operating system on Intel(R) Core(TM) Quad CPU Q6600 with 2GB ofRAM. The webcam Logitech Quickcam Vision Pro is used to capture imagesand is capable of detecting known patterns from a single image and calculatinga 3D position and orientation for world-space and virtual objects (furniture,partition, wall, door etc.) are superimposed on marker tracking.


Some current marker tracking patterns such as ARToolKit library offer veryprecise and robust pattern detection. From the ARToolKit library, we usemk_patt.exe file to produce various ARToolKit from the blankPatt.gif of patternsdirectory. For implementation, several marker patterns and sub-marker forTangible AR were made before the pilot study commenced.

3.5 OCCLUSION BASED INTERACTION -TANGIBLE AR

In this paper we apply two approaches to 2D interactions: a pointer centeredview and an interaction object centered view. In the pointer centered view, thesystem tracks the movement of a single pointer and checks whether there is aninteraction object beneath the pointer. This approach works well in a traditionaldesktop graphical user interface. However, it is not easy to apply the samemethod to Tangible AR environments where natural interaction methods arevital. In the real world, humans are able to use a variety of objects or even theirbare fingers as a pointer. In addition, for some situations such as having multipleparticipants, or using bi-manual interactions, interaction can even involvemultiple pointers.

Detecting pointers over an interaction object can be achieved in numerousways. Detecting occlusions of tracked objects is a passive way to detect pointingactions. Occlusions of interaction objects can be easily utilized as an interactionmethod in Tangible AR environments in which a camera is already availablefor providing real world views to the users, and for tracking the objects ofinterest with passive formal markers.

For occlusion detection, predefined formal markers are widely used fortracking real objects in Tangible AR environments. The vision based trackingsystems usually use multiple markers for tracking one object. A number ofmarkers are attached on a single object in a pre-configured spatial relationship.In this way the object can be tracked successfully even if some of the markersin the marker set are not visible. In addition because the spatial relationships ofall the markers are known, the poses of markers that are not visible can beestimated using the markers that are recognized.

A simple way to guarantee that a marker is within the view volume is tocheck the visibility of its neighboring markers. We refer to these neighbormarkers as boundary markers, and a marker being checked for occlusion as aninteraction marker. To guarantee that an interaction marker is within the viewvolume, boundary markers must be carefully placed. The convex hull ofboundary markers must include the interaction marker. For instance, for a singleinteraction marker, we need at least 2 boundary markers surrounding theinteraction marker (Figure 7). By checking if these boundary markers are visible,the interaction marker can be guaranteed to be within the view volume; hence,it is occluded if it is not detected.


Figure 7. Boundary markers around interaction markers

When multiple interaction markers are placed in a line, neighbors of theinteraction marker being tested can also be treated as boundary markers. Werefer to these markers as hybrid markers (Figure 8). The tested marker is withinthe view volume whenever there is at least one visible boundary (or hybrid)marker on each side. Hybrid markers act as both boundaries and an interactionpoint itself. Occlusion of multiple consecutive markers can also be detected,as well as allowing the boundary markers to be out of the view.

Figure 8. Hybrid markers: the center marker acts as boundary marker for the left marker.

Although the boundary marker method is simple to implement and worksreliably, marker wastage is unavoidable since additional non-interactiveboundary markers are required. Also, interaction is a little difficult because theuser has to ensure enough boundary markers are within the current view.

3.6 INTERACTIVE FURNITURE BY TANGIBLE AUGMENTED REALITY

In this experiment, we modify virtual furniture by occlusion-based interfacesfor Tangible AR effects. Tangible AR interfaces are those in which each virtualobject is registered to a physical object and the user interacts with virtual objectsby manipulating the corresponding physical objects. Using occlusion is a simpleway of finishing interactions by hiding the formal markers being tracked. Twomore sub-marker band cards were made, so each one assumes to control coloror material of virtual furniture. Especially, these marker templates are combinedfrom several unit markers (Figure 9).


Figure 9. Two Sub-marker band cards for Tangible AR control

Each unit marker corresponds to one option. The user takes the first sub-marker band card and reach to virtual chair. The user can control one finger tohide one unit marker. One new corresponded color is assigned to virtual chair.Next, the user moves second sub-marker band card reach to first one, this newone for adjust volume color. So the corresponding virtual color slide is overlaidon this band card. The changing effect of virtual color slide is redefined basedon the position of hidden unit marker using a finger.

4. Application experiment of AR in Interior Design

First, the user prints out a marker that comes with the system. The style andsize of the marker can be defined from the user interface in order to adapt it tothe environment (viewing distance and size of the room). The marker is thenplaced on the floor or wall of the room to be decorated. The user walk aroundin the room and, on the way, takes a series of capture marker images with thedigital camera. The marker images are uploaded to AR software as markertracking. Next, the furniture augmenting system is started.

Our system includes functions for handling images, moving wrl-modelsand sizes, defining marker properties and threshold values and for manipulatingobjects. The user select different pieces of (virtual) furniture from the objectlist at left, then add, delete, modify properties and hide them as required. Eachobject first appears on the marker card, the user can move it to the desiredposition by dragging it with mouse, or modify threshold values of coordinate.Many AR applications use fixed directions in marker coordinates. So whenlooking from the opposite direction, the object would move in an unnaturaldirection. Our approach is more natural for users, as they do not need to knowanything about the marker coordinate. First, the virtual chair and meeting tableare assigned as main sample for our AR experiment. Once the virtual furniture


has been arranged the user may adjust the scale through digital images onscreen or using control marker band through Tangible AR effect. On AR video,the man kept the control marker card on his hand and he reaches towards thevirtual furniture. The virtual adjusted slide appears on the control marker card,so he may interact with virtual furniture through working with this controlmarker card (unit markers must be hidden by finger).

Figure 10. User adjusts the colour of virtual furniture through Tangible AR.

In another phase, the user wants to place one sample partition in the room.A designer may order a virtual partition appear, to move it to the right position.There are problems, the camera would find it difficult to view the trackingmarker when several virtual and real furniture objects are added in. Ourimplementation allows user to quickly change three dimensions relation betweenvirtual furniture and marker images. Thus, the user can place markers on anyfree position so that the camera can view it clearly. As with all virtual furniture,this partition or any virtual furniture component would modify properties inreal time by Tangible AR effect.

Figure 11. Move the chair and add the statue on the table by AR Interior Design interface

All the images and virtual models are loaded to the system dynamically.Furthermore, the state of the virtual furnishing design can be saved in a projectfile which can be loaded later as user desires to continue working with thisdesign.


5. Conclusion

In this paper, virtual furniture and adjustment work was reviewed in regard tofinding a new design method and representation of AR for commercial interiordesign. AR technology opens up new research fields, especially in engineeringand architecture, allowing design work to become more lively, more convenientand intelligent. Design work and manufacturing can be operated at the sametime and they can be more tightly integrated. Furthermore, AR technology canbe employed in and on real sites and users could see furniture through HMD,video display, PDA, etc. It is also thought that the increased interactive potentialwill see increasingly user-centric applications.

Acknowledgement

This work was supported by the Brain Korea 21 Project in 2009.

References

ARToolKit. http://www.hitl.washington.edu/artoolkit

Billinghurst, M., Kato, H. and Poupyrev, I.: 2001, The Magic Book-Moving Seamlessly betweenReality and Virtual, IEEE Computer Graphics and Application.

Billinghurst, M., Kato, H. and Poupyrev, I.: 2001, The MagicBook- Moving Seamlessly betweenReality and Virtual, IEEE Computer Graphics and Application 21(3), 6-8.

Billinghurst, M., Poupyrev, I., Kato, H., and May, R.: 2000, Mixing Realities in Shared Space:An Augmented Reality Interface for Collaborative Computing, Proceeding of ICME 2000,IEEE, 1641.

Dias, J.M.S., Santos, P. and Nande, P.: 2003, In Your Hand Computing: Tangible Interfaces forMixed Reality, Proceedings CD of 2

nd IEEE International Augmented Reality ToolKit

Workshop, Waseda University, Tokyo.

Lee, G.A., Billinghurst, M and Kim, G.J.: 2004, Occlusion based Interaction Methods for tangibleAugmented Reality environments, Proceedings of the 2004 ACM SIGGRAPH internationalconference on Virtual Reality continuum and its applications in industry, Singapore, pp.419- 426.

Viet, T.P, Yeon, C.S., Hak, W.S. and Choi, A.: 2008, AR: Application for prevention of firedisaster in Korean Traditional Buildings, DMACH2008 Conference, CSAAR, Jordan.

virtual reality and simulation...

Documents