convr 2009 proceedings

CONVR 2009 Proceedings of the 9

th International Conference on

Construction Applications of Virtual Reality

Sydney, Australia, 5-6 November 2009

Edited by

Xiangyu Wang

The University of Sydney, Australia

Ning Gu

The University of Newcastle, Australia

Sponsors

The University of Sydney

The CONVR 2009 Conference Organising Committee

Xiangyu Wang, The University of Sydney, Australia

Ning Gu, The University of Newcastle, Australia

Michael Rosenman, The University of Sydney, Australia

Anthony Williams, The University of Newcastle, Australia

Nashwan Dawood, The University of Teesside, United Kingdom

ISBN 978-1-74210-145-3

All rights reserved

© 2009

No part of this work may be reproduced, stored in a retrieval system, or transmitted in any

form or by any means, electronic, mechanical, photocopying, microfilming, recording or

otherwise, without written permission from the Publisher, with the exception of any material

supplied specifically for the purpose of being entered and executed on a computer system,

for exclusive use by the purchaser of the work.

Published and printed at The University of Sydney, Australia, by University Publishing Service.

The University of Newcastle

The University of Teesside

FORUM8 CO.

TABLE OF CONTENTS

PREFACE vii

CONVR 2009 INTERNATIONAL SCIENTIFIC COMMITEE viii ACKNOWLEDGEMENTS ix

KEYNOTE SPEECH ABSTRACTS AND SPEAKER BIOS

Construction Synthetic Environments--------------------------------------------------------1

Simaan M. AbouRizk

Seeking How Visualization Makes Us Smart------------------------------------------------3

Phillip S. Dunston

intuBE Project-------------------------------------------------------------------------------------4

Nashwan Dawood

The Role of VR in Improving Collective Intelligence for AEC Processes-------------6

Mary Lou Maher

I. DESIGN COLLABORATION

Virtual Worlds and Tangible Interfaces: Collaborative Technologies

that Change the Way Designers Think-----------------------------------------------------9

Mary Lou Maher, Ning Gu and Mijeong Kim

A Novel Camera-based System for Collaborative Interaction with

Multi-dimensional Data Models--------------------------------------------------------------19

Michael Van den Bergh, Jan Halatsch, Antje Kunze, Frédéric Bosché, Luc Van Gool and Gerhard Schmitt

Towards a Collaborative Environment for Simulation Based Design----------------29

Michele Fumarola, Stephan Lukosch, Mamadou Seck and Cornelis Versteegt

Empirical Study for Testing Effects of VR 3D Sketching on

Designers’ Cognitive Activities----------------------------------------------------------------39

Farzad Pour Rahimian and Rahinah Ibrahim

Analysis of Display Luminance for Outdoor and Multi-user Use ---------------------49

Tomohiro Fukuda

A Proposed Approach to Analyzing the Adoption and Implementation of

Virtual Reality Technologies for Modular Construction--------------------------------59

Yasir Kadhim, Jeff Rankin, Joseph Neelamkavil and Irina Kondratova

Collaborative 4D Review through the Use of Interactive Workspaces---------------71

Robert Leicht and John Messner

Design Scenarios: Methodology for Requirements Driven

Parametric Modelling of High-rises---------------------------------------------------------79

Victor Gane and John Haymaker

An Experimental System for Natural Collocated and Remote Collaboration--------91

Jian Li and Jingyu Chen

Urban Wiki and VR Applications--------------------------------------------------------------97

Wael Abdelhameed and Yoshihiro Kobayashi

II. AUTOMATION AND INTERACTION

Toward Affective Handsfree Human-machine Interface Approach in

Virtual Environments-based Equipment Operation Training--------------------------107

Iman Mohammad Rezazadeh, Xiangyu Wang, Rui Wang and Mohammad Firoozabadi

Construction Dashboard: An Exploratory Information Visualization Tool

for Multi-system Construction----------------------------------------------------------------117

Cheng-Han Kuo, Meng-Han Tsai, Shih-Chung Kang and Shang-Hsien Hsieh

Computer Gaming Technology and Porosity-----------------------------------------------127

Russell Lowe and Richard Goodwin

Virtual Reality User Interfaces for the Effective Exploration and

Presentation of Archaeological Sites----------------------------------------------------------139

Daniel Keymer, Burkhard Wünsche and Robert Amor

Interactive Construction Documentation----------------------------------------------------149

Antony Pelosi

Case Studies on the Generation of Virtual Environments of

Real World Facilities-----------------------------------------------------------------------------155

Michele Fumarola and Ronald Poelman

Evaluation of 3D City Models Using Automatic Placed Urban Agents----------------165

Gideon Aschwanden, Simon Haegler, Jan Halatsch, Rafaël Jeker, Gerhard Schmitt

and Luc van Gool

Integration of As-built and As-designed Models for

3D Positioning Control and 4D Visualization during Construction---------------------177

Xiong Liang, Ming Lu and Jian-Ping Zhang

Augmenting Site Photos with 3D As-built Tunnel Models for

Construction Progress Visualization----------------------------------------------------------187

Ming Fung Siu and Ming Lu

Automatic Generation of Time Location Plan in Road Construction Projects------197

Raj Kapur and Nashwan Dawood

Development of 3D-Simulation Based Genetic Algorithms to

Solve Combinatorial Crew Allocation Problems-------------------------------------------207

Ammar Al-Bazi, Nashwan Dawood and John Dean

Integration of Urban Development and 5D Planning--------------------------------------217

Nashwan Dawood, Claudio Benghi, Thea Lorentzen and Yoann Pencreach

III. SIMULATION AND ANALYSIS

A Simulation System for Building Fire Development and

the Structural Response due to Fire----------------------------------------------------------229

Zhen Xu, Fangqin Tang and Aizhu Ren

Physics-based Crane Model for the Simulation of Cooperative Erections-----------237

Wei Han Hung and Shih Chung Kang

Interaction between Spatial and Structural Building Design:

A Finite Element Based Program for the Analysis of

Kinematically Indeterminable Structural Topologies-------------------------------------247

Herm Hofmeyer and Peter Russell

Virtual Environment on the Apple iPhone/iPod Touch-----------------------------------257

Jason Breland and Mohd Fairuz Shiratuddin

3D Visibility Analysis in Virtual Worlds: The Case of Supervisor---------------------267

Arthur van Bilsen and Ronald Poelman

Evaluation of Invisible Height for Landscape Preservation

Using Augmented Reality-----------------------------------------------------------------------279

Nobuyoshi Yabuki, Kyoko Miyashita and Tomohiro Fukuda

An Experiment on Drivers’ Adaptability to Other-hand Traffic

Using a Driving Simulator----------------------------------------------------------------------287

Koji Makanae and Maki Ujiie

C2B: Augmented Reality on the Construction Site----------------------------------------295

Léon van Berlo, Kristian Helmholt and Wytze Hoekstra

Development of a Road Traffic Noise Estimation System

Using Virtual Reality Technology-------------------------------------------------------------305

Shinji Tajika, Kazuo Kashiyama and Masayuki Shimura

Application of VR Technique to Pre- and Post-Processing for

Wind Flow Simulation in Urban Area--------------------------------------------------------315

Kazuo Kashiyama, Tomosato Takada, Tasuku Yamazaki, Akira Kageyama,

Nobuaki Ohno and Hideo Miyachi

Construction Process Simulation Based on Significant Day-to-day Data-------------323

Hans-Joachim Bargstädt and Karin Ailland

Effectiveness of Simulation-based Operator Training------------------------------------333

John Hildreth and Michael Stec

IV. BUILDING INFORMATION MODELLING

BIM Server: Features and Technical Requirements--------------------------------------345

Vishal Singh and Ning Gu

LEED Certification Review in a Virtual Environment-----------------------------------355

Shawn O’Keeffe, Mohd Fairuz Shiratuddin and Desmond Fletcher

Changing Collaboration in Complex Building Projects

through the Use of BIM-------------------------------------------------------------------------363

Saskia Gabriël

The Introduction of Building Information Modelling in Construction Projects:

An IT Innovation Perspective-----------------------------------------------------------------371

Arjen Adriaanse, Geert Dewulf and Hans Voordijk

Creation of a Building Information Modelling Course for

Commercial Construction at Purdue University------------------------------------------383

Shanna Schmelter and Clark Cory

vii

Preface

The Faculty of Architecture, Design and Planning, at the University of Sydney and the

School of Architecture and Built Environment, at the University of Newcastle are proud to

co-host CONVR 2009, the 9th

International Conferences on Construction Applications of

Virtual Reality. Significantly, the conference is the 9th

gathering of this international body of

scholars and professionals across all Architecture, Engineering and Construction (AEC)

disciplines, who dedicate and contribute to the knowledge building and applications of a

broad range of advanced visualisation technologies in the AEC industry. Although the name

of the conference has “Virtual Reality” (VR) as the keyword, it actually covers much broader

range of visualisation–related topics beyond VR, which makes the range of the audiences

wider with broader impact as years go.

The CONVR 2009 conference has attracted much attention and recognition among the

research and professional communities involved in the AEC industry. The organising

committee received close to 70 abstracts. After two rounds of rigorous double blind reviews

(the first round for abstract review and the second round for full paper review) by

International Scientific Committee, the CONVR 2009 is very pleased to accept 39 high

quality full papers in this volume.

The CONVR 2009 conference provides a unique platform for experts in the fields to report,

discuss and exchange new knowledge, which has resulted from the most current research

and practice of advanced visualisation technologies. The Organising Committee is pleased

to present selected papers that highlight the state-of-the-art development and research

directions across the following four themes:

• Design Collaboration• Automation and Interaction

• Simulation and Analysis

• Building Information Modelling

In the following pages, you will be able to find a range of quality papers that truly capture

the quintessence of these concepts and will certainly challenge and inspire readers.

The CONVR 2009 Conference Organising Committee:

Xiangyu Wang (Chair), The University of Sydney, Australia

Ning Gu (Co-chair), The University of Newcastle, Australia

Michael Rosenman (Co-chair), The University of Sydney, Australia

Anthony Williams, The University of Newcastle, Australia

Nashwan Dawood, The University of Teesside, United Kingdom

November 2009

viii

CONVR 2009 International Scientific Committee

Karin Ailland Bauhaus-University Weimar

Robert Amor The University of Auckland

Serafim Castro The University of Teesside

Chiu-Shui Chan Iowa State University

Clark Cory Purdue University

Robert Cox Purdue University

Nashwan Dawood The University of Teesside

Paulo Dias Instituto de Engenharia Electrónica e Telemática de Aveiro (IEETA)

Ning Gu The University of Newcastle, Australia

Fátima Farinha EST- Algarve University Portugal

Michele Fumarola Delft University of Technology

Jan Halatsch ETH Zurich

David Heesom The University of Wolverhampton

Wei-Han Hung National Taiwan University

Rahinah Ibrahim University Putra Maylaysia

Vineet Kamat The University of Michigan

Jeff Kan Taylor College Malaysia

Shih-Chung Kang National Taiwan University

MiJeong Kim Kyung Hee University

Robert Lipman National Institute of Standards and Technology (NIST)

Russell Lowe The University of New South Wales

Koji Makanae Miyagi University

John Messner Penn State University

Esther Obonyo The University of Florida

Svetlana Olbina The University of Florida

Aizhu Ren Tsinghua University

Enio Emanuel Ramos Russo Catholic University in Rio

Iman Rezazadeh Islamic Azad University

Michael Rosenman The University of Sydney

Marc Aurel Schnabel Hong Kong Chinese University

Mohd Fairuz Shiratuddin The University of Southern Mississippi

Augusto de Sousa Universidade do Porto

Andrew Strelzoff Brown University

Walid Tizani The University of Nottingham

Xiangyu Wang The University of Sydney

Vaughn Whisker Penn State University

Antony Williams The University of Newcastle, Australia

ix

ACKNOWLEDGEMENTS

We express our gratitude to all authors for their enthusiasms to contribute their research as

published in this proceedings. Furthermore, this proceedings would not have been possible

without the constructive comments and advice from all the International Scientific

Committee members. We are also deeply grateful to the other members on the organising

committee, Dr. Michael Rosenman, Professor Anthony Williams, and Professor Nashwan

Dawood. Thanks and appreciation specifically goes to Ms Rui Wang for designing our

proceedings and CD covers. We are also grateful to the conference assistants Ms Mercedes

Paulini, Mr Lei Hou and Mr Wei Wang, whose great backup support is essential for the

success of the conference. Financial aid came from Design Lab at the Faculty of

Architecture, Design and Planning at the University of Sydney, School of Architecture and

Built Environment at the University of Newcastle Australia, and Forum8 Co.

KEYNOTE SPEECH 1

Dr. Simaan M. AbouRizk, Professor and NSERC Industrial Research Chair in Construction Engineering

and Management Canada Research Chair in Operation Simulation Department of Civil and Environmental

Engineering, University of Alberta, Canada.

“Construction Synthetic Environments”

The presentation describes our vision for a highly

integrated, interoperable, distributed simulation

framework for modeling and analyzing construction

projects.

We first describe the evolution of simulation

applications (over a period of 15 years) within the

construction industry in Alberta by providing the

attendees with an overview of select implementation

of simulation-based systems in industrial

applications. Systems were introduced through

collaborations between major construction companies

and the University of Alberta. Those systems were

deployed by the partner companies in different ways,

including planning for tunnel construction projects,

scheduling of modules in a module yard with space

constraints for an industrial contractor, analysis of

fabrication shops for improvement, process

improvement studies, and others.

The presentation then provides an overview of our

vision of advanced simulation systems we call

Construction Synthetic Environments (COSYE), the

intent of which will be to achieve “a fully integrated, highly automated construction execution environment

across all project phases and throughout the facility’s life cycle”, as articulated in Figure 1. The figure

demonstrates a large-scale distributed simulation framework that provides a comprehensive representation of

an entire construction project with all of its components, including: a model of the facility (product model), the

production/construction operations (process models), the business models, the resources involved, and the

environment under which the project takes place. The framework allows the simulation models to extend

throughout the life of the project with real-time input and feedback to manage the project until it is handed over

to operations. The goal is to provide a virtual world where a construction project is planned, executed, and

controlled with minimum disruption to the actual project. The framework will provide means to establish:

“detailed and comprehensive modeling of the entire life cycle of facilities;

collaboration amongst a variety of stakeholders in building the required virtual

models that represent the project; seamless integration between various forms of

simulation (discrete, continuous, heuristic, etc.) and simulation software and tools;

reusable simulation components for many applications (e.g. weather generation,

equipment breakdown processes etc); and man-machine interactions with the

models.”

We have completed three prototype synthetic environments using the COSYE framework over the past few

years, including ones for industrial construction, steel construction, a bidding game, and tunnel construction.

We will provide an overview of these during the presentation and select environment to demonstrate in greater

detail.

1

The Construction Synthetic Environment Framework

Bio of Dr. AbouRizk:

Dr. AbouRizk currently holds the positions of “Canada Research Chair in Operation Simulation” and the

“Industrial Research Chair in Construction Engineering and Management” in the Department of Civil and

Environmental Engineering at the University of Alberta. He received his PhD degree from Purdue University

in 1990 and his MSCE from Georgia Tech in 1985. He joined the University of Alberta in 1990 and was

promoted to full professor in July 1997.

Dr. AbouRizk’s research accomplishments have been recognized through numerous awards for the quality of

his research in the field of construction engineering and management, including the prestigious ASCE Peurifoy

Construction Research Award, the E.W.R. Steacie Memorial Fellowship from the Natural Sciences and

Engineering Research Council of Canada, the Thomas Fitch Rowland Prize for best paper in construction

engineering management, the Killam Professorship, the Walter Shanly Award, and the E. Whitman Wright

Award.

Dr. AbouRizk has led the development of the Hole School of Construction Engineering at the University of

Alberta into one of the most reputable construction engineering and management programs in North America,

boasting global recognition for the success of its graduate students and the strength of its faculty members. The

success and distinctiveness of this program are based on strong industry collaboration in the areas of research,

teaching, and overall practice. Dr. AbouRizk’s method has garnered wide support from funding agencies,

policy makers, and industry practitioners, and has attracted some of the brightest students from around the

world. He is renowned in the academic construction community for his research in computer simulation and its

applications in construction planning, productivity improvement, constructability reviews and risk analysis.

2

KEYNOTE SPEECH 2

Dr. Phillip S. Dunston, Associate Professor in the Division of Construction Engineering and Management,

School of Civil Engineering, at Purdue University, USA.

“Seeking How Visualization Makes Us Smart”

It was in 1996 that a computer science researcher

brought a vision for applying Augmented Reality

visualization technology to the attention of

attendees at a civil engineering computing

conference. The door was then opened for a new

set of inquiring minds to join architects who were

already taking a look at the possibilities of virtual

visualization. The exciting visualization

opportunities presented by Virtual Reality and

Mixed Reality technologies have since captured the

attention of a growing number of researchers from

the broad architecture, engineering, construction

and facilities management (AEC/FM) domain.

Unlike computer science and computer engineering

researchers who have a technology development

perspective, the AEC/FM community has a user

perspective that must be developed as part of our

contribution to shaping the development of these

technologies and ultimately realizing their adoption

into practice. Opportunities exist for improving

practice through new efficiencies and through

devising new ways of executing work tasks. Our

attention to human resource capabilities and the

attendant human factors can yield successful technology development decisions and integration. This keynote

talk will review our experience in exploring this softer side as well as how we have inevitably had to confront

the more technical challenges and will also suggest how some future objectives might be pursued.

Bio of Dr. Dunston:

Phillip S. Dunston, an Associate Professor with appointments in the Division of Construction Engineering and

Management and the School of Civil Engineering at Purdue University in West Lafayette, Indiana, USA. He is

a 2003 US National Science Foundation Career grantee for research on Mixed Reality applications for the

architecture, engineering and construction (AEC) industry. He directs the Advanced Construction Systems

Laboratory (ACSyL) and is a Co-Director of the Center for Virtual Design of Healthcare Environments, both at

Purdue. His research emphasizes the human factors related to virtual visualization and applying such principles

in specifying the features and functions of visualization systems.

3

KEYNOTE SPEECH 3

Prof. Nashwan Dawood, Director for the Centre for Construction Innovation & Research,

University of Teesside and Cecil M Yuill Professor of Construction management & IT, UK.

“intUBE Project” Intelligent Use of Building’s

Energy Information (www.intube.eu)

It is a well established fact that buildings are one of the

major contributors to energy use and CO2 emissions.

The energy used in buildings accounts for 40 % of the

total energy use in Europe. While some breakthroughs

are expected in new buildings, the pace of these

improvements is too slow considering the EU's

ambitious goal to improve energy efficiency by 20 %

before 2020. With over 80% of the European buildings

standing in 2020 being already built, the main aim of

the IntUBE project is to develop and make use of

information and communications technologies

including Virtual Reality to improve the energy

efficiency of these existing buildings in compliance

with the EU's aims of improving energy efficiency.

IntUBE will develop tools for measuring and analysing

building energy profiles based on user comfort needs.

These will offer efficient solutions for better use and

management of energy use within buildings over their

lifecycles. Intelligent Building Management Systems

will be developed to enable real-time monitoring of

energy use and optimisation. They will, through

interactive visualisation of energy use, offer solutions

for user comfort maximisation and energy use

optimisation.

intUBE concept

4

Bio of Prof. Dawood:

Prof. Dawood is the director for Construction Innovation & Research at the unversity of Teesside and hold

Yuil Professor Chair. Prof Dawood has spent many years as an academic and researcher within the field of

construction management and the application of IT in the construction process. This has ranged across a

number of research topics including information technologies and systems (4D,VR,Integrated databases), risk

management, and business processes. This has resulted in over 170 published papers in refereed international

journal and conferences, and research grants from British Council, Industry, Engineering Academy , EPSRC,

DTI and construction industry companies, totalling about £2,500,000. Final reports of the last three EPSRC

grants received ‘Tending to Outstanding' peer assessment review from EPSRC.

I have been a visiting fellow/Professor at VTT -Finland, University of Calgary- Canada, University of Bahrain-

Bahrain, Central University of Taiwan, AIT- Thailand, Stanford University-USA, PWRI (Public Works

Research Institutes)- Japan, Georgia Tech- USA, Virginia Tech- USA, UNSW- Australia, University of

Parana- Brazil, University of Florida-USA, International Islamic University Malaysia, Gyeongsang National

University, Korea and Miyagi university , Japan and Osaka University, Japan.

Prof. Dawood has originated the CONVR conference series (Construction Applications of Virtual Reality:

Current Initiatives and Future Challenges). The mission of this is to bring together national and international

researchers and practitioners from all areas of the construction industry and promote efficient exchange of

ideas and develop mutual understanding of needs and potential applications of VR modelling. CONVR 2000

was organised at Teesside and attended by participants from 9 countries, CONVR 2001 organised at Chalmers

University, Sweden and attended by participants from 12 countries. CONVR 2003 was organised by Virginia

Tech, USA, CONVR 2004 was organised by ADETTI (Portugal), CONVR 2005 organised in Durham UK,

CONVR 2006 was organised by Florida State University and CONVR 2007 was organised at Penn State

University, USA, CONVR 2008 was organised by IIUM Malaysia and CONVR 2009 will be organised by the

University of Sydney, Australia.

5

KEYNOTE SPEECH 4

Prof. Mary Lou Maher, the Deputy Division Director of the Information and Intelligent Systems Division

at National Science Foundation and Professor at the University of Sydney, Australia.

“The Role of VR in Improving Collective Intelligence for AEC Processes”

Collective intelligence is a kind of intelligence that

emerges from the collaboration and competition of

individuals. While the concept of collective intelligence

has been around for a long time, recent renewed interest

in collective intelligence is due to internet technologies

that allow collective intelligence to emerge from

remotely located and potentially very large numbers of

individuals. Wikipedia is a product of collective

intelligence as a source of knowledge that is

continuously generated and updated by very large

numbers of individuals. Similarly, Second Life is a

product of collective intelligence as a 3D virtual world

that is created and modified by the large numbers of

individuals that enter the world. The AEC industry relies

on the collective intelligence of many individuals and

teams of professionals from different disciplines. Virtual

reality in its many forms provides collaborative

technologies that not only enable people to work

together from a distance but also change the way we

interact with each other and the shared digital models

that comprise the product of the collaboration. This

presentation presents a new kind of collective

intelligence enabled by emerging technologies, the

research challenges, and the potential impact on design

and creativity in AEC projects.

Bio of Prof. Maher:

Mary Lou Maher is the Deputy Division Director of the Information and Intelligent Systems Division at NSF.

She joined the Human Centered Computing Cluster in July 2006 and initiated a funding emphasis at NSF on

research in creativity and computing called CreativeIT. She is the Professor of Design Computing at the

University of Sydney. She received her BS (1979) at Columbia University and her MS (1981) and PhD (1984)

at Carnegie Mellon University. She was an Associate Professor at Carnegie Mellon University before joining

the University of Sydney in 1990. She has held joint appointments in the Faculty of Architecture and the

School of Information Technologies at the University of Sydney. Her own research includes empirical studies

and new technologies for design in virtual worlds and other collaborative environments, behavior models for

intelligent rooms, motivated reinforcement learning for non-player characters in MMORPGs, and tangible user

interfaces for 3D design.

6

DESIGN COLLABORATION

Virtual Worlds and Tangible Interfaces: Collaborative Technologies

That Change the Way Designers Think-----------------------------------------------------9

Mary Lou Maher, Ning Gu and Mijeong Kim

A Novel Camera-based System for Collaborative Interaction with

Multi-dimensional Data Models--------------------------------------------------------------19

Michael Van den Bergh, Jan Halatsch, Antje Kunze, Frédéric Bosché, Luc Van Gool and Gerhard Schmitt

Towards a Collaborative Environment for Simulation Based Design----------------29

Michele Fumarola, Stephan Lukosch, Mamadou Seck and Cornelis Versteegt

Empirical Study for Testing Effects of VR 3D Sketching on

Designers’ Cognitive Activities----------------------------------------------------------------39

Farzad Pour Rahimian and Rahinah Ibrahim

Analysis of Display Luminance for Outdoor and Multi-user Use ---------------------49

Tomohiro Fukuda

A Proposed Approach to Analyzing the Adoption and Implementation of

Virtual Reality Technologies for Modular Construction--------------------------------59

Yasir Kadhim, Jeff Rankin, Joseph Neelamkavil and Irina Kondratova

Collaborative 4D Review through the Use of Interactive Workspaces---------------71

Robert Leicht and John Messner

Design Scenarios: Methodology for Requirements Driven

Parametric Modelling of High-rises---------------------------------------------------------79

Victor Gane and John Haymaker

An Experimental System for Natural Collocated and Remote Collaboration-------91

Jian Li and Jingyu Chen

Urban Wiki and VR Applications-------------------------------------------------------------97

Wael Abdelhameed and Yoshihiro Kobayashi

9th International Conference on Construction Applications of Virtual Reality Nov 5-6, 2009

VIRTUAL WORLDS AND TANGIBLE INTERFACES: COLLABORATIVE TECHNOLOGIES THAT CHANGE THE WAY DESIGNERS THINK

Mary Lou Maher, Professor,

University of Sydney, Australia;

[email protected], http://web.arch.usyd.edu.au/~mary

Ning Gu, Lecturer,

University of Newcastle, Australia;

[email protected], http://www.newcastle.edu.au/school/arbe

Mi Jeong Kim, Lecturer,

Kyung Hee University, Korea;

[email protected], http:// housing.khu.ac.kr

ABSTRACT: Reflecting on the authors’ computational and cognitive studies of collaborative design, this paper

characterizes recent research and applications of collaborative technologies for building design. The specific

technologies considered are those that allow synchronous collaboration while planning, creating, and editing 3D

models, including virtual worlds, augmented reality (AR) tabletop systems, and tangible user interfaces (TUIs).

Based on the technical capabilities and potential of the technologies described in the first part, the second part of

the paper considers the implications of these technologies on collaborative design based on an overview of the

results of two cognitive studies conducted by the authors. Two studies, using protocol analysis, are described as the

basis for characterizing the designers’ cognitive actions, communication and interaction in different collaborative

design situations. The first study investigated collaborative design in a virtual world to better understand the

changes in design behavior when the designers are physically remote but virtually collocated as avatars in a 3D

model of their design solution. The second study measured the effects of tangible user interfaces (TUIs) with AR on

a tabletop system on designers’ cognitive activities and design process in co-located collaboration. The paper

concludes by discussing the implications of the results of these studies on the future design of collaborative

technologies for designers.

KEYWORDS: Collaborative Design, 3D Virtual Worlds, Tangible Interfaces, Protocol Analysis, Design Cognition.

1. COLLABORATIVE TECHNOLOGIES AND DESIGN

Collaborative design is a process of dynamically communicating and working together within and across disciplines

in order to collectively establish design goals, search through design problem spaces, determine design constraints,

and construct a design solution. While each designer contributes to the development of the design solution,

collaboration implies teamwork, negotiation, and shared models. Collaborative technologies support design in

several ways, two of which are (1) the ability for collaboration to occur at the same time while the participants are

remotely located and (2) the ability to augment the perception of the shared design drawings or models through new

technologies for interacting with digital models. In this paper we show how two specific collaborative technologies

change the way designers think.

We focus on the architectural design of buildings where eventually the design solution is a model of a 3D product

that evolves as the record and the focus of the design process. Bringing designers into 3D virtual environments has

the potential to improve their understanding of the design models during the collaborative process. Two such

environments, 3D virtual worlds and tangible user interfaces (TUIs) to 3D models, are very different approaches to

making the design model accessible to remote and collocated designers. Various virtual worlds and tangible

interaction technologies have been developed for the AEC (Architecture, Engineering and Construction) domain, but

most of them are still in the lab-based prototype development and validation stages. In this paper we reflect on the

potential and implications of these collaborative technologies from a cognitive perspective in order to understand

their role in design practice and to contribute a cognitive basis for the design of new collaborative technologies.

9


2. BACKGROUND

The background section reviews recent developments in collaborative technologies and introduces a research

method - protocol analysis - for studying design cognition in collaborative design.

2.1 Developments in collaborative technologies for design

Collaborative technologies for design allow two or more designers to create shared drawings or model and design

together while remote or collocated. Since the focus while designing is on the shared drawings or models of the

design, the important aspects of collaborative technologies for design are: the type of digital media available to

represent the design, the interaction technologies for creating, visualizing, and modifying the shared drawings or

models, and the ways in which the designers communicate and interact with each other.

Research into digital technologies for supporting collaborative work started in the 1960s with the early work at

Stanford Research Institute into innovative interaction techniques. Later the developments at Xerox PARC in the

1970s and 1980s brought the field to what became known as user-centered design (Norman and Draper 1986) an

important research focus in an emerging field called Human Computer Interaction (HCI). In the early 1990s, a

branch of HCI developed into the research area of Computer Supported Cooperative Work (CSCW) and groupware

technologies. Like HCI, CSCW is a multi-disciplinary field that has made significant contributions to the design of

collaborative technologies. All these technological developments changed the practice of many fields such as

science, art and design. In terms of functions, the developments in computer-supported technologies for

collaborative design can be classified into the categories of video conferencing, shared drawings, and shared models.

The types of digital media for design representation can include bit mapped images, sketches, structured graph

models, 2D and 3D geometric CAD models, and 3D object oriented models. Some commonly used technologies for

supporting the collaborative design of buildings are digital sketching tools such as GroupBoard

(http://www.groupboard.com), Sketchup (http://sketchup.google.com), and the major CAD systems. The interaction

technologies include the standard graphical user interfaces (GUI) using keyboard and mouse, touch screens and

tables such as the Microsoft Surface (http://www.microsoft.com/surface/), augmented reality, and game controllers

such as the Nintendo Wii (http://wii.com). Designers can communicate with each other using text chat, voice over

IP, and/or video. With the technical advances in the field and the wider adoption of high-bandwidth internet, new

generation of collaborative technologies have emerged, two of which including 3D virtual worlds and TUIs to 3D

models are the focus of our paper.

3D virtual worlds are networked environments designed using the place metaphor. One of the main characteristics

that distinguish 3D virtual worlds from conventional virtual reality is that 3D virtual worlds allow multiple users to

be immersed in the same environment supporting a shared sense of place and presence (Singhal and Zyda 1999).

Multi-user 3D virtual worlds have grown very rapidly, with examples such as Second Life

(http://www.secondlife.com) having reached millions of residents and boasting a booming online economy. Through

the use of the place metaphor, 3D virtual worlds have been associated with the physical world ever since the early

conceptual formation of the field. On one hand, the rich knowledge and design examples in the physical world

provide a good starting point for the development of 3D virtual worlds. On the other hand, designing in 3D virtual

worlds is becoming an exciting territory for the new generation of designers to explore. For the AEC industry, recent

developments in 3D virtual worlds and the proliferation of high bandwidth networked technologies have shown

great potential in transforming the nature of remote design collaboration. In 3D virtual worlds, designers can

remotely collaborate on projects without the barriers of location and time differences. With high-speed network

access, real-time information sharing and modifications of large data sets such as digital building models become

possible over the World Wide Web. Distant design collaboration can significantly reduce the relocation costs and

help to increase efficiency in global design firms. Current development of such systems, for example, DesignWorld

(Maher et al. 2006a) supports remote communication, collaborative 3D modeling and multidisciplinary building

information sharing.

TUIs couple physical artifacts and architectural surfaces to the correlated digital information, whereby the physical

representations of digital data serve simultaneously as interactive controls. For design applications, TUIs are often

combined with AR, which allows designers to explore design alternatives using modifiable models. Through the

direct tangible interaction, designers can use their hands and often their entire bodies for the physical manipulation.

More recently, a wide range of tangible input devices have been developed for collaborative design (Deitz and Leigh

10


2001; Fjeld et al. 1998; Ishii and Ullmer 1997; Moeslund et al. 2002). BUILD-IT provides physical 3D ‘bricks’ for

concurrently accessing to multiple digital models and the ARTHUR system employs a ‘wand’ for interacting with

3D virtual settings projected into their common working environment. The metaDESK embodies many of the

metaphorical devices of windows, icons, and menus of GUIs as a physical instantiation such as a lens with wooden

frames, phicons, and trays respectively. The InteracTable and DiamondTouch use touch-sensitive displays for

supporting cooperative team work. These tangible input devices support multi-user interactions for co-located or

remote collaboration by providing a sensory richness of meaning. Tangible interaction is recently becoming an

important research issue in the domain of design computation. However, compared to the rapid development of

tangible interaction technologies, relatively little is known about the effects of tangible interaction systems on design

cognition.

2.2 Studying design cognition in collaborative design: protocol analysis

While usability is a critical feature of collaborative technologies, usability studies do not necessarily provide a

critical assessment of the impact of technology on design processes. Rather than focus on usability, studying the

perception, actions, and cognition of designers using collaborative technologies allows us to compare the

technologies and their impact on human behavior while designing. Protocol analysis has been used to study how

people solve problems, and has been used widely in the study of design cognition (Gero and Mc Neill 1997; Suwa et

al. 1998). We adapt this method to study perception, action, and cognition while designers are using collaborative

technologies.

A protocol is the recorded behavior of the problem solver which is usually represented in the form of sketches,

notes, video or audio recordings. Whilst the earlier studies dealt mainly with protocols’ verbal aspects, later studies

acknowledge the importance of design drawing, associating it with design thinking which can be interpreted through

verbal descriptions (Stempfle and Badke-Schaub 2002; Suwa and Tversky 1997). Recent design protocol studies

employ analysis of actions which provide a comprehensive picture of physical actions involved during design

(Brave et al. 1999). In design research, two kinds of protocols are used: concurrent protocols and retrospective

protocols. Generally, concurrent protocols are collected during the task and utilized when focusing on the process-

oriented aspect of designing, being based on the information processing view (Simon 1992). The ‘think-aloud’

technique is typically used, in which subjects are requested to verbalize their thoughts as they work on a given task

(Ericsson and Simon 1993; Lloyd et al. 1995). Retrospective protocols are collected after task and utilized when

focusing on the content-oriented aspects of design, being concerned with the notion of reflection in action (Dorst

and Dijkhuis 1995; Schön 1983).

The methodology involves developing an experiment in which one or more designers are asked to work on a design

task while being recorded. The recording, or protocol data, is a continuous stream of data and can include video of

the designers, and/or continuous video of the computer display showing the designers’ actions and an audio stream

of the verbalization of the designer(s). The protocol data is segmented into units that are then coded and analyzed to

characterize the design session. The coding scheme is developed according to the theory, model, or framework that

is being tested and can include cognitive, communication, gesture, or interactive actions.

The protocol analysis technique has been adopted to understand the interactions of design teams (Cross and Cross

1996; Stempfle and Badke-Schaub 2002) and design behavior of teams (Goldschmidt 1996; Valkenburg and Dorst

1998). Protocol studies of collaborative architectural design focus on understanding team collaboration, in terms of

use of communication channels and design behavior variables (Gabriel and Maher 2002). Protocol coding has been

conducted on professional architects and students architects respectively for the two studies described below in

Sections 3 and 4. For the two studies, the think aloud method is not directly applicable in the protocol collection.

The protocol data comprises the designers’ conversations, gestures, and interactions rather than the designers’

verbalization of their thoughts as in the think aloud method. Such collaborative protocols provide data indicative of

cognitive activities that are being undertaken by the designers, not interfering with design process as a natural part of

the collaborative activities.

3. DESIGNING IN A 3D VIRTUAL WORLD

This study compares the collaborative design process in a 3D virtual world to collaborative design processes in a

traditional face-to-face sketching environment and in a remote sketching environment. We set up three distinctive

11


design environments and studied four pairs of professional architects1 each collaborating on a different design task

with similar complexity in each of the three design environments. The comparison of the same pairs of designers in

three different environments is assumed to provide a better indication of the impact of the environments on design

cognition than using different designers or a same design task. A more detailed description this study is reported in

(Maher et al. 2006b).

3.1 Experiment setup and coding scheme design

A different design brief and a collage of the photos showing the site and the surroundings were provided for each of

the three different experiment sessions. In the face to face session of the experiment, the pairs of designers used pen

and paper as the sketching tools. In the remote sketching session, they sketched remotely and synchronously, with

one designer using a Smart Board (http://www.smarttech.com) and the other using Mimio (http://www.mimio.com).

Both technologies provide a pen and digital ink interface. In the final 3D virtual world session, designers

collaborated remotely and synchronously in a 3D virtual world - Active Worlds (http://www.activeworlds.com) -

through 3D design and modeling. In the latter two sessions of the experiment, remote and synchronous

communication was simulated by locating both designers in two different parts of the same room, allowing them to

talk to each other, but only seeing each other via web cams. Each session required the designers to complete the

design task in 30 minutes. They were given training sessions on the use of Smart Board, Mimio and Active Worlds

prior to the experiment.

The basis of the coding scheme design for the research is a consideration of a set of expected results. We developed

and applied a five-category coding scheme including communication content, design process, operations on external

representations, function-structure, and working modes. The communication content category partitions each

session according to the content of the designers’ conversations, focusing on the differences in the amount of

conversation devoted to discussing design development when compared to other topics. The design process category

characterizes the different kinds of designing tasks that dominate in the three different design environments. The

operations on external representation category look specifically at how the designers interacted with the external

design representation to see if the use of 2D sketches or 3D models was significantly different. The function-

structure category further classifies the design-related content as a reference to the function of the design or the

structure of the design. The working modes category characterizes each segment according to whether or not the

designers were working on a same design task or on a same part of the design representations.

3.2 Protocol analysis result

The analysis of collaborative design behavior involves documenting and comparing the categories of codes. We

looked at frequencies of the occurrence of the code categories in the three different sessions. We also documented

the time spent for each category, with respect to the total time elapsed during the session. This data gives us the

duration percentages of the codes in each main category. Table 1 provides an overview of the focus of activity in

each of the three design environments by showing the average percentages for 4 of the 5 coding categories of the

pairs of the designers who participated in the experiment. We don’t show the working mode category in TABLE 1

because it will always be a total of 100% of the duration since the designers are always either working on the same

or different tasks. The categories of codes were applied independently therefore each segment could be coded in

more than one category.

TABLE. 1: Durations of codes in each main category as average percentages of the total elapsed time.

Categories Face to Face Sketching Remote Sketching 3D Virtual Worlds

Communication Content 72% 72% 61%

Design Process 69% 48% 34%

Operations on External Representations 96% 90% 93%

Function-Structure 67% 43% 27%

1 While four pairs of designers are not considered a statistically significant number of participants in a cognitive study, we can

use these results as an exploratory study to identify major differences that are common across this sample of designers.

12


We tested if there are significant differences between the pairs across the three different design sessions in terms of

the design behavior (coded activity categories). The ANOVA test (ANOVA with replication, P<0.05 between the

three different design sessions) result shows that there is no significant difference between the pairs in terms of

communication content (p=0.15), operations related to external representations (p=0.80) and working mode

(p=0.99). The results are listed in TABLE 2. These results support that the collaborative behavior in these categories

did not show a significant variance amongst the different pairs. Note that the design process and function-structure

categories are significantly different between the pairs. These differences are not surprising and they are common in

design studies since design activities of a particular designer can change due to different situations in the actual

context, and the variance in individual design strategies can have an effect on the collaborative design process.

TABLE. 2: ANOVA test result on action categories of different designer pairs.

P-value

Communication

Content

Design

Process

Operations on External

Representations

Function-

Structure

Working

Mode

Between designer pairs in each design session 0.15 2.46E-08 0.80 3.26E-05 0.99

The details of the protocol analysis, reported in (Maher et al. 2006b), show that there are insignificant differences

among the three design environments, in terms of the communication content and operations on external

representations categories. 3D virtual worlds are able to support design communication and representation during

collaboration. There is a significant decrease from face to face sketching to remote sketching and to 3D design and

modeling in virtual worlds, in terms of the design process and function-structure categories. The first interpretation

of such differences is that during remote collaboration, designers are able to design both collectively and

individually due to the flexibility of digital media in modifying and integrating different parts or different versions

of the design representations, as well as the physical separation of the designers. The evidence for this is the results

of the working mode category, showing a significant difference among the face to face and remote design

environments. It was observed in our experiment that during the face to face sessions, designers mostly worked

together with over 95% of the duration devoted to collaborative mode. Although sometimes a particular designer led

the process while the other observed and critiqued, they always focused on the same tasks. In the remote sessions,

and especially in 3D virtual worlds, an average 40% of the duration was for individual design phases where different

designers worked on different tasks or different parts of the design representations. They often came together after

an individual phase to review each other’s outcomes or swap tasks. During these individual phases, they reduced and

some pairs even stopped verbal communications (note the decrease in the communication content category for 3D

virtual worlds). This explains the smaller number of design-related segments during the remote sketching and 3D

virtual world sessions. The second interpretation of such differences is the possible changes in the approach to

design development when switching from 2D sketching to 3D modeling. In sketching sessions, although designers

constantly externalize their design ideas in separated sketch parts or over other existing sketches, there is usually a

clear separation between the development of design concepts and the development of formal design representations.

In 3D virtual worlds, these two processes become more blurry. The 3D virtual world objects designers used to

explore concepts also become the 3D models for the final design representations. The design ideas are evolved,

explored and externalized all through 3D modeling. Our current perceptions about 3D modeling and the current

setups of the experiment are inadequate to further understand the different roles of 3D modeling in design

collaboration. Future research is needed in this regard.

A further analysis on each of the coded categories was also conducted and the main points are summarized below:

(1) the communication about awareness of the other designer increases during remote collaboration with the highest

duration percentage observed in 3D virtual worlds. There is a growing focus on the communication about design

representations but the changes across the three design environments are not significant; (2) the highest percentage

of visual analysis for design development is observed in 3D virtual worlds. 3D modeling as the main design

approach in virtual worlds does not fit into the traditional “analysis-synthesis” model and should be studied further;

(3) As discussed, in virtual worlds designers use 3D models for both exploring design concepts and representing

final outcomes and often the transformation of the 3D models captures the design development process. The most

frequent operations on external representations are change-related activities in 3D virtual worlds, while create-

related activities occur most frequently in the two sketching environments.

Both similar and different patterns were observed when designers change from face to face collaboration to remote

collaboration, and from 2D sketching to 3D modeling. In a follow-up study, we further observed and surveyed the

13


same designer pairs’ experiences and preferences when they were given the choice to collaborate using the typical

3D modeling features supported in virtual worlds or the integrated remote sketching features. It was noted that (1)

different designers can have different preferences in applying 2D sketching, 3D modeling or the combination of the

two for design collaboration; (2) 3D models appear to be a more popular media for final design representations

compared to 2D sketches (3) both 2D sketches and 3D models can be used to explore abstract design concepts such

as spatial volumes and relationships as well as to develop external representations such as design details; and (4) 3D

virtual worlds support both collaborative and individual design tasks while remote sketching environments tend to

encourage collaborative tasks.

4. DESIGNING USING TANGIBLE USER INTERFACES

Until recently, research on TUIs focused on developing new systems and exploring technical options rather than

addressing users’ interaction experience in the new hybrid environments. This study explored the potentials and

perspectives of tangible interaction with an AR display for supporting design collaboration by measuring the effects

of TUIs on designers’ cognitive activities and design process. The aim was to gain a deeper understanding of the

designers’ experience of tangible interaction specifically, in terms of the collaborative affordance of TUIs, in the

context of co-design. In this study, physical manipulation and spatial interaction were considered as tangible

interaction. Physical manipulation of objects exploits intuitive human spatial skills, where movement and perception

are tightly coupled (Hornecker and Buur 2006; Sharlin et al. 2004). Spatial interaction is engaging in interaction

with the space created by the spatial arrangement of the design objects, and the use of this engagement for

communication in collaboration.

4.1 Experiments and coding scheme design

A tabletop system with TUIs including a horizontal surface and a vertical display has been developed at the Design

Lab at the University of Sydney to support problem solving, negotiation and establishing shared understanding in

collaborative design. As multiple, specialized tangible input devices for TUIs, 3D blocks with tracking markers in

ARToolKit (Billinghurst et al. 2001) can be attached to different functions, each independently accessible to 3D

virtual objects (Fitzmaurice 1996). The tabletop system was compared to a typical desktop system with GUIs for 3D

spatial planning tasks in a controlled laboratory experiment. Multiple designers can concurrently access the 3D

blocks whereas only one person can edit the model at a time using the mouse and keyboard. Comparing

conventional input devices such as mouse and keyboard, it was assumed that the affordances of the physical handles

of the TUIs facilitate two-handed interactions and thus offers significant benefits to collaborative working in design

collaboration (Granum et al. 2003; Moeslund et al. 2002)

The participants comprised three pairs of 2nd or 3rd year architecture students, with minimum of one year’s

experience as CAD users. Each pair performed both sessions, a TUI and a GUI, in one day for two different design

tasks. The chosen scenario was the redesign of a studio into a home office or a design office by configuring spatial

arrangements of furniture, where each 3D block can represent a piece of furniture in the tabletop system, and pre-

designed furniture can be imported from the library in ArchiCAD using a mouse and keyboard. Two design tasks

were developed to be similar in complexity and type, and the systems were assessed by letting designers discuss the

existing design and proposed new ideas in co-located collaboration. Designers’ conversation, interactions, and

gestures were videotaped while they designed the layout for four required areas according to the design

requirements and then the analysis of the data carried out using the protocol analysis method. The collected data in

the form of collaboration protocols were transcribed and then segmented using an utterance based technique. Each

utterance flagged the start of a new segment, and then for each segment relevant codes were assigned according to a

customized coding scheme.

The coding scheme comprised six categories at four levels: 3D modeling actions at the Action level, perceptual

activities at the Perceptual level, set-up goal activities and co-evolution at the Process level, and cognitive

synchronization and gesture actions at the Collaborative level. The Action level represents designers’ tangible

interaction with the external representation through the 3D modeling actions. The Perception level represents

designers’ perception of visuo-spatial features in the external representation. Designers’ perceptual activities in 3D

configurations are related to the reconsideration for different meaning and function of the same objects rather than

reconstructing unstructured images. The Process level represents designers’ ‘problem-finding’ behaviors associated

with creative design. Specifically set-up goal activities refer to introducing new functional issues as new design

14


requirements. The co-evolution represents the exploration of two design spaces, problem and solution spaces. The

Collaborative level reflects designers’ collective cognition. Cognitive synchronization represents the argumentative

process in collaborative design, showing how designers construct a shared understanding of the problem and a

shared representation of the solution through negotiating activities. Gesture actions represent non-verbal design

communication in collaborative design. More information on the coding scheme can be found in Kim’s PhD thesis

(Kim 2006).

4.2 Protocol analysis results

TABLE 3 shows the mean values of segment durations in the design sessions ranging from 6.9 to 8.9 seconds. The

standard deviations are rather high (5.0 to 7.2 seconds), which suggests that the distribution of the segment durations

is widespread from the mean values. The average segment duration in TUI sessions (7.3 second) was shorter than

that in the GUI (8.3 second) sessions. Thus designers’ utterances during TUI sessions were on average shorter than

those in GUI sessions. Designers in the GUI session did not make as much progress in developing design solutions

as they did in the TUI sessions given the same amount of time.

TABLE. 3: Duration of segments

Pair 1 Pair 2 Pair 3

Session/Design task TUI1/A GUI2/B TUI2/A GUI1/B TUI1/A GUI2/B

Task completion Yes Yes Yes No Yes No

Total time 1065 sec 1116 1206 sec 1196 sec 892 sec 911 sec

Segment no. 133 80 89 66 120 81

Mean (sec) 7.3 8.9 6.9 7.3 7.8 8.8

Std. Deviation 4.8 6.6 4.9 7.2 5.4 5.8

Session: 1– first session; 2 – second session / Design task: A - Home office; B – Design office

The difference in the task progression might also have affected the occurrence of cognitive actions as shown in

TABLE 4. With the direct, naïve manipulability of physical objects and rapid visualization, designers in the TUI

session produced more multiple cognitive actions compared to designers in the GUI session (210 and 127). The

average occurrence of the perceptual actions in the TUI session (105) was twice that of the GUI session (57), and the

average occurrence of set-up goal actions in the TUI session (36) was almost twice that of the GUI session (20). In

order to identify how the different HCI environments influenced the proportion of the cognitive actions, the

relatively higher occurrences of each action category are shaded for each designer. Since we intended to identify the

trend in the differences between the two design sessions through the results of the collaborative study, we assumed

that a value higher than 2% between the two design sessions would indicate the changes according to the different

interaction modes. The trend shows that all designers produced more perceptual actions (50.0% and 44.9%) in the

TUI session, and more functional actions (32.9% and 39.4%) in the GUI session.

TABLE 4. Occurrence percentages of action categories

Perceptual Functional Set-up goal Total actions

TUI 136 (52.9%) 81 (31.5%) 40 (15.6%) 257 (100%) Pair 1

GUI 57 (39.0%) 63 (43.2%) 26 (17.8%) 146 (100%)

TUI 100 (46.9%) 77 (36.2%) 36 (16.9 %) 213 (100%) Pair 2

GUI 61 (45.2%) 53 (39.3%) 21 (15.5%) 135 (100%)

TUI 77 (49.4%) 54 (34.6%) 25 (16.0%) 156 (100%) Pair 3

GUI 50 (47.6%) 39 (37.2%) 16 (15.2 %) 105 (100%)

TUI average 105 (50.0%) 69 (32.9%) 36 (17.1%) 210 (100%) Pairs (average)

GUI average 57 (44.9%) 50 (39.4%) 20 (15.7%) 127 (100%)

With the focus being on designers’ spatial cognition, six cognitive action categories were investigated in terms of

four levels of the coding scheme. The encoded protocols were analyzed using a Mann-Whitney U test to examine

differences between the two sessions, and the structures of design behaviors were explored through the interactive

graphs. 3D modeling actions, perceptual and set-up goal activities were combined into generic activity components,

highlighting the different patterns of design behaviors. In order to compare in a same condition for two design

sessions, the total time of each GUI session was cut at a same time point as the corresponding TUI session.

15


Epistemic action refers to ‘exploratory’ motor activity, where users would get the final stage through performing a

lot of physical actions even with no specific goal (Fitzmaurice and Buxto 1997; Kirsh and Maglio 1994). The

findings at the Action level suggested that in order to develop design ideas designers in the TUI session changed the

external representation by performing more ‘movement’ modeling actions rather than compute it mentally. The short

and frequent 3D modeling actions are considered as epistemic actions, reducing their cognitive load and producing a

‘conversation’ style of interaction, to express and negotiate design ideas, establishing shared understanding in design

collaboration. These epistemic 3D modeling actions allowed designers to test their ideas quickly by proceeding in

small steps and then these continuities brought more opportunities to execute creative leaps in designing. The

findings at the Perception level imply that by manipulating 3D blocks, designers produced more perceptual actions,

whereby more new visuo-spatial features were created and discovered through performing these modeling actions.

The new perceptual information could have brought about new interpretations on the external representation since

thinking about emergent properties evokes shifting focus to a new topic. Furthermore, designers using TUIs

perceived more new and existing ‘spatial relationships’ among elements, while focusing more on the ‘existing’

elements themselves in the GUI session. Perceiving more ‘spatial relationships’ potentially encouraged designers to

explore related functional thoughts, to go beyond retrieving the visual information, and thus make abstract

inferences.

The findings at the Process level suggested that designers invented new design requirements by restructuring the

problem space based on the perceived information in a situated way rather than synthesizing design solutions fixed

on the initial requirements. The retrieval of knowledge, based on their expertise or past experience, for the new

constraints suggests that designers’ recall might be improved through the manipulation of 3D blocks. Furthermore,

they developed the formulation of the problem and alternatives for a solution throughout the design session, showing

a co-evolutionary process which is associated with creative outcomes in designing. Consequently, designers using

these 3D blocks would gain more opportunities to discover key concepts through the ‘problem-finding’ process. The

embodied facilitation theme highlights that tangible interaction embodies structure that allows or hinders some

actions, and thereby shapes emerging group behavior. Spatial interaction refers to the fact that tangible interaction is

embedded in real space, thus has the potential to employ full-body interaction, acquiring communicative function.

The findings from the Collaborative level reveal that designers in the TUI session tend to establish more cognitive

synchronization through active negotiation processes, especially, the cycle of three codes ‘Propose’, ‘Argument’ and

‘Resolution’. The horizontal table and 3D blocks on the tabletop system might facilitate collaborative interactions by

working as the embodied structure. Spatial interaction with spaces was also facilitated while using 3D blocks, thus

designers in the TUI session produced more immersive gesture actions using hands and arms, leading to whole body

interaction with the external representation. ‘Touch’ actions seemed to be beneficial for designers’ perceptual

activities because designers in the TUI sessions kept touching the 3D blocks, which might have simplified

designers’ mental computation through epistemic actions.

To sum up, the protocol analysis of the four levels of designers’ spatial cognition reveals that the epistemic 3D

modeling actions using TUIs put much less load on designers’ cognitive processes, thus resulting in the co-

generation of new conceptual thoughts and perceptual discoveries in the external representation. The off-loaded

designers’ cognition affected the design process by increasing ‘problem-finding’ behaviors associated with creative

design and supported design communication, negotiation and shared understanding. In conclusion, the results of this

study suggest that tangible interaction afforded by TUIs provides important benefits for designers’ spatial cognition

and a cognitive and collaborative aid for supporting collaborative design. To verify these conclusions, more

designers need to be observed and their protocols analyzed.

5. IMPACT OF COLLABORATIVE TECHNOLOGIES ON DESIGN

In this paper we describe collaborative technologies as enablers of collective intelligence in design in two ways: (1)

the ability for collaboration to occur at the same time while the participants are remotely located and (2) the ability

to augment the perception of shared design drawings or models through new technologies for interacting with digital

models. Since the focus while designing is on the shared drawings or models of the design, critical aspects of

collaborative technologies for design are: the type of digital media available to represent the design, the interaction

technologies for creating, visualizing, and modifying the shared drawings or models, and the ways in which the

designers communicate and interact with each other.

16


In our study of 3D virtual worlds, we show how a multi-user virtual world that allows co-editing of 3D models

changes the behavior of designers in two important ways: given the same amount of time for the collaborative

session, the designers worked on the same task most of the time while collocated and only part of the time when

remotely located in physical space but collocated in the virtual world; and the 3D model as a focus for design

development and communication embodied concept and geometry in the 3D objects where this appears to be

separated in sketches. These results show that the type of digital media available, comparing 2D sketches and 3D

models, changes what designers talk about when they are collaborating; and the change from physically located

around a shared drawing to remotely located within a 3D model changes the working modes and encourages

designers to smoothly move between working on the same task and working on different aspects of the design.

In our study of TUIs compared to GUIs, we show how tangible interaction on a tabletop encourages the designers to

engage in more exploratory design actions, and more cognitive actions in general. We attribute this change to the

additional perceived affordances of the tangible blocks as interfaces to the digital model when compared to the

keyboard and mouse as the interface to the digital model. The blocks became specific parts of the model for the

designers as they moved the pieces on the tabletop, while the keyboard and mouse was negotiated to be different

parts of the model at different times. These affordances effectively allowed the designers to focus their actions

directly on the design alternatives and development rather than on the interface to the design model. This difference

affected the number of segments, an increased number when using TUIs implying an epistemic approach to

generating alternatives, and the number of cycles in the problem-finding process implying a potential for more

creative solutions.

The results from these two very different studies converge in a set of recommendations for the design of

collaborative technologies for designers: The development of multi-user 3D virtual worlds for designers, when

merged with current CAD capabilities, has the benefit of a smooth transition between working on the same task and

working on separate tasks. When using CAD systems for collaboration, a significant amount of time is spent

coordinating working on the same or different tasks at same or different times. A 3D virtual world enables a group

of designers to be more aware of each other’s presence when they are focused on designing. While the main role of

3D modeling in traditional CAD systems is design documentation, our study shows that 3D modeling can play

different roles in design collaboration from the early concept exploration to the final design representation. The

development of TUIs for 3D modeling changes the perception and interaction with digital models and should

become a standard alternative to the keyboard and mouse for design. Most collaborative technologies for designers

still rely on the keyboard and mouse for interacting with digital models. There is significant benefit and relatively

little training needed to incorporating TUIs into 3D modeling systems so that designers can choose the best

interaction style for the stage of the design and modeling tasks.

6. REFERENCES

Billinghurst, M., Kato, H., and Poupyrev, I. (2001). "MagicBook: a Transitional AR Interface." Computer Graph 25,

745-753.

Brave, S., Ishii, H., and Dahley, A. (1999). "Tangible Interface for Remote Collaboration and Communication." CHI

'99: Conference on Human Factors in Computing Systems, Pittsburgh, Pennsylvania, USA, 394-401.

Cross, N., Christiaans, H and Dorst, K (1996). Analysing Design Activity, John Wiley & Sons Ltd, Chichester, UK.

Cross, N., and Cross, A. (1996). "Observations of Teamwork and Social Processes in Design." Analysing Design

Activity, N. Cross, H. Christiaans, and K. Dorst, eds., John Wiley & Sons Ltd, Chichester, UK.

Deitz, P., and Leigh, D. (2001). "DiamondTouch: A Multi-User Touch Technology." UIST 2001: The 14th Annual

ACM Symposium on User Interface Software and Technology, Orlando, Florida, USA.

Dorst, K., and Dijkhuis, J. (1995). "Comparing Paradigms for Describing Design Activity." Design Studies 16(2),

261-275.

Ericsson, K., and Simon, H. (1993). Protocol Analysis: Verbal Reports as Data, MIT Press, Cambridge,

Massachusetts, USA.

Fitzmaurice, G. (1996). "Graspable User Interfaces", University of Toronto, Toronto.

17


Fitzmaurice, G., and Buxto, W. (1997). "An Empirical Evaluation of Graspable User Interfaces: Towards

Specialized, Space-multiplexed Input." CHI '97: Conference on Human Factors in Computing Systems, Los

Angeles, USA, 43-50.

Fjeld, M., Bichsel, M., and Rauterberg, M. (1998). "BUILD-IT: an Intuitive Design Tool based on Direct Object

Manipulation." Gesture Workshop '96: Gesture and Sign Language in Human-Computer Interaction,

Bielefeld, Germany, 297-308.

Gabriel, G., and Maher, M. (2002). "Coding and Modeling Communication in Architectural Collaborative Design."

Automation in Construction 11, 199-211.

Gero, J., and Mc Neill, T. (1997). "An Approach to the Analysis of Design Protocols." Design Studies 19(1), 21-61.

Goldschmidt, G. (1996). "The Designer as a Team of One." Analysing Design Activity, N. Cross, H. Christiaans,

and K. Dorst, eds., John Wiley & Sons, Chichester, UK.

Granum, E., Moeslund, T., and Störring, M. (2003). "Facilitating the Presence of Users and 3D Models by the

Augmented Round Table." Presence 2003: The 6th Annual International Workshop on Presence 19.

Ishii, H., and Ullmer, B. (1997). "Tangible Bits: Towards Seamless Interfaces between People, Bits and Atoms."

CHI '97: Conference on Human Factors in Computing Systems, Los Angeles, USA, 234-241.

Kim, M. (2006). "Effects of Tangible User Interfaces on Designers' Spatial Cognition," University of Sydney,

Sydney.

Kirsh, D., and Maglio, P. (1994). "On Distinguishing Epistemic from Pragmatic Action." Cognitive Science 18, 513-

549.

Lloyd, P., Lawson, B., and Scott, P. (1995). "Can Concurrent Verbalization Reveal Design Cognition?" Design

Studies 16 (2), 237-259.

Maher, M., Rosenman, M., Merrick, K., and Macindoe, O. (2006a). "DesignWorld: An Augmented 3D Virtual

World for Multidisciplinary Collaborative Design." CAADRIA 2006, Osaka, Japan, 133-142.

Maher, M. L., Bilda, Z., Gu, N., Gul, F., Huang, Y., Kim, M. J., Marchant, D., and Namprempree, K. (2006b).

"CRC Design World Report." University of Sydney, Sydney.

Moeslund, T., Störring, M., and Liu, Y. (2002). "Towards Natural, Intuitive and Non-Intrusive HCI Devices for

Roundtable Meetings." MU3I: Workshop on Multi-User and Ubiquitous User Interfaces, Funchal, Madeira,

Portugal, 25-29.

Norman, D., and Draper, S. (1986). "User Centered System Design: New Perspectives on Human-Computer

Interaction." Lawrence Erlbaum Associates, Hillsdale, NJ.

Schön, D. (1983). The Reflective Practitioner: How Professionals Think in Action, Basic Books, New York.

Sharlin, E., Watson, B., Kitamura, Y., Kishino, F., and Itoh, Y. (2004). "On Tangible User Interfaces, Humans and

Spatiality." Personal and Ubiquitous Computing 8(5), 338-346.

Simon, H. (1992). The Sciences of the Artificial, The MIT Press, Cambridge, Massachusetts, USA.

Singhal, S., and Zyda, M. (1999). Networked Virtual Environments: Design and Implementation, ACM Press, New

York.

Stempfle, J., and Badke-Schaub, P. (2002). "Thinking in Design Teams - an Analysis of Team Communication."

Design Studies 23(5), 473-496.

Suwa, M., Purcell, T., and Gero, J. (1998). "Macroscopic Analysis of Design Processes based on A Scheme for

Coding Designers' Cognitive Actions." Design Studies 19(4), 455-483.

Suwa, M., and Tversky, B. (1997). "What Do Architects and Students Perceive in Their Design Sketches?" Design

Studies 18(4), 385-403.

Valkenburg, R., and Dorst, K. (1998). "The Reflective Practice of Design Teams." Design Studies 19(3), 249-271.

18


A NOVEL CAMERA-BASED SYSTEM FOR COLLABORATIVE INTERACTION WITH MULTI-DIMENSIONAL DATA MODELS

*

Michele Van den Bergh,

Computer Vision Laboratory, ETH Zurich, Zurich, Switzerland;

[email protected]

Jan Halatsch,

Chair of Information Architecture, ETH Zurich, Zurich, Switzerland;

[email protected]

Antje Kunze,


[email protected]

Frédéric Bosché, PhD,


[email protected]

Luc Van Gool, Prof.,


[email protected]

Gerhard Schmitt, Prof.,


[email protected]

ABSTRACT: In this paper, we address the problem of effective visualization of and interaction with multiple and

multi-dimensional data supporting communication between project stakeholders in an information cave. More

exactly, our goal is to enable multiple users to interact with multiple screens from any location in an information

cave. We present here our latest advancements in developing a novel human-computer interaction system that is

specifically targeted towards room setups with physically spread sets of screens. Our system consists of a set of

video cameras overseeing the room, and of which the signals are processed in real-time to detect and track the

participants, their poses and hand-gestures. The system is fed with camera based gesture recognition. Early

experiments have been conducted in the Value Lab, which has been introduced recently at ETH Zurich, and they

focus on enabling the interaction with large urban 3D models being developed for the design and simulation of

future cities. For the moment, experiments consider only the interaction of a single user with multiple layers (points

of view) of a large city model displayed on multiple screens. The results demonstrate the huge potential of the

system, and the principle of vision based interaction for such environments. The work continues on the extension of

the system to a multi-user level.

KEYWORDS: Information cave, interaction, vision, camera, hand gestures.

1. INTRODUCTION

1.1 Product Information Models for Design and Simulations

Future cities, standing for evolving medium-size and mega-cities, have to be understood as a dynamic system – a

network that bridges different scales, such as local, regional, and global scales. Since such a network comprises

several dimensions, for example social, cultural, and economic dimensions it is necessary to connect active research,

*This work was supported by the Competence Center for Digital Design Modeling (DDM) at ETH Zurich.

19


project management, urban planning as well as communication with the public to establish a mutual vision, or to

map the desires of the involved participants.

In the last few decades, the use of computers, software and digital models has expanded within many fields related

to the Architecture, Engineering, Construction and Facility Management (AECFM), where Facility may refer to

commercial, industrial or infrastructure building assets, and also cities. However, it is only recently that researchers

have started tackling the problems of the compartmentalization of this expansion within these different fields

corresponding to the multiple stakeholders of such project. And, this expansion occurred without wider project

integration. For example, in urban planning, multiple different digital models are often used to perform different

analyses such as: CO2 emissions, energy consumption and traffic load. Nonetheless, significant progresses have

recently been made in the integration of information models into what are now commonly referred to Building

Information Models (BIM), City Information Models (CIM), etc.

These integrated models enable earlier and more systematic (sometimes automated) detection of conflicts different

multiple analysis and processes. However, the resolution of these conflicts still requires human negotiations, and

effective methods and technologies for interacting collaboratively with the information in order to resolve detected

conflicts are still missing. The main complexity here is that large projects, such as large scale planning projects,

require the involvement of many technical experts and other stakeholders (e.g. owners, pubic) who approach

projects from many different view points, which results in many different types of conflicts that must resolve

collaboratively.

In order to address this problem, holistic participative planning paradigms (governing process management, content

creation as well as design evaluation) have to evolve, and consider new software and hardware solutions that will

enable the different stakeholders to effectively work collaboratively.

1.2 Example: Dübendorf Urban Planning Project

Today’s urban planning and urban design rely mainly on static representations (e.g. key visuals, 3D models). Since

the planning context and its data (for example scenario simulations) are dynamic, visual representations need to be

dynamic and interactive too, resulting in the need for physical environments enabling such dynamic processes.

During spring semester 2009 students researched how to establish design proposals in a more collaborative manner.

The focus was on an urban planning project, the rehabilitation of the abandoned Swiss military airport in Dübendorf.

The main goal of this research project was to develop an interactive shape grammar model (Müller, 2009), which

was implemented with the CityEngine (http://www.procedural.com/cityengine). In combination with real-time

visualization using Autodesk Showcase (http://usa.autodesk.com/adsk/servlet/index?id=6848305&siteID=123112),

a better understanding design interventions was achieved.

While this research project showed the feasibility of collaborative interactive design, the experiments, then

conducted in the Value Lab (see section 2) showed that the interactivity offered by such information caves did not

always meet the expectations of the users (see analysis in section 2).

FIG. 1: As a result of collaborative city design workshops a new use for an abandoned military airport in the out-

skirts of Zurich had been implemented with the collaborative interaction tools that are available at the Value Lab.

20


1.3 Information Visualization Caves

Information visualization caves have been investigated in order to enable stakeholders to sit in a single room and

collaboratively solve conflicts, during planning, construction or operation. Such caves are typically designed with

complex multimedia settings to enable participants to visualize the project model, and the possible conflicts at hand,

from multiple points of view simultaneously (e.g. owner vs. user vs. contractor, contractor A vs. contractor B).

Traditional human-computer interaction devices (e.g. mouse, keyboard) are typically focused to fulfill single user

requirements, and are not adapted to work with the multiplicity of participants and the multi-dimensionality (as well

as multiplicity) of the data sets representing large projects. Solutions have however been proposed to improve

interactivity. A multi-screen setup can drastically enhance collaboration and participatory processes by keeping

information present to all attendees, and such setup is common in information caves (Gross et al., 2003, König et al.,

2007). Additionally, (multi-) touch screens are now available as more intuitive multi-user human-computer

interaction devices. However, despite their definite advantages for interactions with multiple users, particularly in

table settings, multi-touch screens remain inadequate for use in rooms with physically spread sets of screens, as they

require the users to constantly move from a screen to the other.

2. VALUE LAB

The ETH Value Lab (see figure 2) is a special kind of information visualization room, and was designed as a

research platform to guide and visualize long-term planning processes while intensifying the focus on the

optimization of buildings and infrastructures through new concepts, new technologies and new social behaviors to

cut down CO2 emissions, energy consumption, traffic load, and to increase the quality of life in urban environments

(Halatsch and Kunze, 2007). It helps researchers and planners to combine existing realities with planned

propositions, and overcome the multiplicity (GIS, BIM, CAD) and multi-dimensionality of the data sets representing

urban environments (Halatsch et al., 2008a and 2008b).

The Value Lab consists of a physical space with state-of-the art hardware (supercomputer), software (e.g. urban

simulation and CAD/BIM/GIS data visualization packages) and intuitive human-computer interaction devices. The

interface consists of several high-resolution large area displays including:

• Five large screens with a total of 16 mega pixels and equipped with touch interface capabilities; and

two; and

• Three FullHD projectors. Two projectors form a concatenated high-resolution projection display with

4 Megapixel in resolution. That particular configuration is for example used for real-time landscape

visualization. The third projector delivers associated views for videoconferencing, presentation and

screen sharing.

The computing resources, display and interaction system produces a tremendous amount of possible configurations

especially in combination with the connected computing resources. The system manages all computing resources,

operation systems, displays, inputs, storage and backup functionality in the background as well as lighting

conditions and different ad hoc user modes.

As a result, The Value Lab forms the basis for knowledge discovery and representation of potential transformations

of the urban environment, using time-based scenario planning techniques in order to test the impact of varying

parameters on the constitution of cities. It shows how the combination of concepts for hardware, software and

interaction can help to manage digital assets and simulation feedback as well as promoting visual insights from

urban planners to associated stakeholders in a human-friendly computer environment (Fox, 2000).

However, as discussed earlier, we found out that beside the direct on-screen manipulation of information, a

technology was needed to steer larger moderated audiences inside a project, and that offers a more integrated

navigation and usability behavior as well as permitting a wider overview on the main contents presented.

Therefore we are investigating a novel touch-less interaction system with camera-based gesture recognition. This

system is presented below and early experimental results are presented in section 4.

21


FIG. 2: The Value Lab represents the interface to advanced city simulation techniques and acts as the front-end of

the ETH Simulation Platform.

3. VISION SYSTEM

In this section, we describe the vision system, that detects and tracks hand gestures of a user in front of a camera

mounted on top of a screen as shown in figure 3. The goal of the system is to enable the interaction of the person

with a 3D model.

A recent review of vision-based hand pose estimation (Erol et al., 2007) states that currently, the only technology

that satisfies the advanced requirements of hand-based input for human computer interaction is glove-based sensing.

In this paper, however, we aim to provide hand-based input without the requirement of such markers.

The first contribution is an improved skin color segmentation algorithm that combines an offline and an online

model. The online skin model is updated at run-time based on color information taken from the face region of the

user. This skin color segmentation is used to detect the location of the hands. The second contribution is a novel

hand gesture recognition system, which combines the classification performance of average neighborhood margin

maximization (ANMM) with the speed of 2D Haarlets. The system is example-based, matching the observations to

predefined gestures stored in a database. The resulting system is real-time and does not require the use of special

gloves or markers.

FIG. 3: Person interacting with a camera and screen.

3.1 Skin Color Segmentation

The hands of the user are located using skin color segmentation. The system is hybrid, combining two skin color

segmentation methods. The first is a histogram-based method, which can be trained online, while the system is

running. The advantage of this system is that it can be adapted in real-time to changes in illumination and to the

person using the system. The second method is trained in advance with a Gaussian mixture model (GMM). The

22


benefit of the offline trained system is that it can be trained with much more training data and is more robust.

Howe`ver, it is not robust to changes in illumination or to changes in the user.

3.1.1 Online model

Every color can be represented as a point in a color space. A recent study (Schmugge et al., 2007) tested different

color spaces, and concluded that the HSI (hue, saturation and intensity) color space provides the highest

performance for a three dimensional color space, in combination with a histogram-based classifier.

A nice characteristic is that the histograms can be updated online, while the system is running. Two histograms are

kept, one for the skin pixel color distribution (Hskin), and one for the non-skin colors (Hnon-skin). For each frame in the

incoming video stream, the face region is found using a face detector such as the one in OpenCV

(http://opencvlibrary.sourceforge.net/), and the pixels inside the face region are used to update Hskin. Then, the skin

color detection algorithm is run and it finds the face regions as well as other skin regions such as the hands and

arms. The pixels that are not classified as skin are then used to update Hnon-skin.

3.1.2 Offline model

In the GMM-based approach, the pixels are transformed to the rg color space. A GMM is fitted to the distribution of

the training skin color pixels using the expectation maximization algorithm as described in (Jedynak et al., 2002).

Based on the GMM, the probabilities P(skin|color) can be computed offline, and stored in a lookup table.

3.1.3 Post processing

On one hand, the histogram-based method performs rather well at detecting the skin color pixels under varying

lighting conditions. However, as it bases its classification on very little input data, it has a lot of false positives. On

the other hand, the GMM-based method performs well in constrained lighting conditions. Under varying lighting

conditions it tends to falsely detect white and beige regions in the background. By combining the results of the

histogram-based and the GMM-based methods, many false positives can be eliminated. The resulting segmentation

is improved further in additional post processing steps, which include median filtering and connected components

analysis.

3.2 Hand Gesture Recognition

The hand gesture recognition algorithm is based on the full body pose recognition system using 2D Haarlets

described in (Van den Bergh et al., 2009). Instead of using silhouettes of a person as input for the classifier, hand

images are used.

3.2.1 Classifier input

The hands are located using the skin color segmentation algorithm described in section 4.1. A cropped grayscale

image of the hand is extracted, as well as a segmented silhouette, which are then concatenated into one input sample,

as shown in figure 4. The benefit of using the cropped image without segmentation, as shown on the right, is that it

is very robust for noisy segmentations. Using the silhouette based on skin color segmentation only, as shown on the

left, the background influence is eliminated. Using the concatenation of both gives us the benefit of both input

sample options.

FIG. 4: Example of an input sample.

3.2.2 Haarlet-based classifier

For details about the classifier we refer to (Van den Bergh et al., 2009). It is based on an average neighborhood

margin maximization (ANMM) transformation T, which projects the input samples to a lower dimensional space, as

23


shown in figure 5. This transformation is approximated using Haarlets to improve the speed of the system. Using

nearest neighbors search, the coefficients are then matched to hand gestures stored in a database.

FIG. 5: Structure of the classifier illustrating the tranformation T (dotted box), approximated using Haarlets. The

Haarlet coefficients are computed on the input sample. The approximated coefficients (that would result from T) are

computed as a linear combination C of the Haarlet coefficients.

4. EXPERIMENTS

In this section, we describe the demo application that allows for the visualization of 3D models that can be loaded

into the program. Using hand gestures, the user can zoom in on the model, pan and rotate it.

4.1 Gestures

The hand gesture classifier is trained based on a set of training samples containing the gestures shown in figure 6.

An example of the system detecting these static gestures is shown in figure 7.

FIG. 6: The gestures that are trained in the hand gesture classifier.

FIG. 7: Examples of the hand gesture recognition system detecting different hand gestures.

The hand gesture interaction in this application is composed of the hand gestures shown in figure 6. It recognizes the

gestures and movements of both hands to enable the manipulation of the object/model. Pointing with one hand

selects the model to start manipulating it. By making two fists, the user can grab and rotate the model along the z-

axis. By making a fist with just one hand, the user can pan through the model. By making a pointing gesture with

both hands, and pulling the hands apart, the user can zoom in and out of the model. The open hands release the

model and nothing happens until the user makes a new gesture. An overview of these gestures is shown in figure 8.

24


(a) select (start interaction)

(b) rotating

(c) panning

(d) zooming

(e) release (do nothing)

FIG. 8: The hand gestures used for the manipulation of the 3D object on the screen.

The time delay of the vision system on average is 31 ms. This is the time recorded from sending the command to

grab the image from the camera, to sending the resulting interaction commands to the user interface. This allows us

to run the system at a refresh rate of 30 Hz. The accuracy of the recognition system (using the three hand poses

shown in figure 6) is 99.8%. Increasing the number of trained poses from three to ten results in a recognition

accuracy of 98.2%. The accuracy of the hand localization based on the skin color segmentation is less than a pixel,

granted that the hand is segmented correctly. The cases where the hand is not segmented are: when the hand

overlaps with the face of the user, or overlaps with a similarly colored person or object in the background. These are

predictable and could be eliminated in future work with a form of depth estimation, of which unfortunately no

accurate real-time implementations exist to our knowledge at time of writing.

4.2 Application

The interaction system above has been implemented as an extension of an open-source 3D model viewer, the GLC

Player (http://www.glc-player.net/). This enables us to: (1) load models in multiple formats (OBJ, 3DS, STL, and

OFF) and of different sizes, and (2) use our hand interaction system in combination with standard mouse and

keyboard interaction. Pressing a button in the toolbar activates the hand interaction mode, after which the user can

start gesturing to navigate through the model. Pressing the button again deactivates the hand interaction model and

returns to the standard mouse-keyboard interaction mode.

We conducted experiments by installing our system in the Value Lab and tested with multiple 3D models, and in

particular with a model created as part of the Dübendorf urban planning project. This model represents an area of

about 0.6 km2 and is constituted of about 4000 objects (buildings, street elements, trees) with a total of about

500,000 polygons. Despite this size, our system achieved frame rates of about 30fps (frame per second), which is

sufficient for smooth interaction. Examples of the user zooming, panning and rotating through the 3D model are

shown in figures 9, 10 and 11 respectively. In each figure, the left column shows side and back views of the system

in operation at the beginning of the gesture, and the right columns the same views but at the end of the gesture.

The hand interaction mode is currently only available for model navigation (rotation, panning and zooming), all the

other features of the viewer being only accessible in mouse-keyboard interaction mode. Nonetheless, our

implementation enables simple extensions of the hand interaction mode. In the near future, we for instance aim to

enable the hand interaction mode for object selection (to view its properties).

25


FIG. 9: Zooming into the model.

FIG. 10: Panning the model.

26


FIG. 11: Rotating the model.

5. CONCLUSION

In this paper, we first described the need for novel human-computer interaction tools, enabling users in information

visualization caves to simultaneously interact with large amounts of information displayed on multiple screens

spread around the cave. Today’s urban design tasks could be significantly enhanced in terms of interaction

especially when different stakeholders are involved. Currently available interaction devices, such as mouse-

keyboard or screen (multi-) touch capabilities, are often not adapted to such requirements, and this was confirmed in

an urban design project conducted in the Value Lab at ETH Zurich.

A novel solution for human-computer interaction was then introduced that is based on vision. Compared to currently

existing systems, it presents the advantage of being marker-less. Experiments, conducted in the Value Lab,

investigated the usability of this system in a situation as realistic as possible. For these, our interaction system has

been integrated to a 3D model viewer, and tested with a large 3D model of an urban development project. The

results show that our system enables a stable, smooth and natural interaction with 3D models at refresh rates of 30

Hz.

Nonetheless, these results remain preliminary. The system is not always as robust as it should be, and its

applicability to enable multiple users to simultaneously interact with multiple screens remains to be demonstrated.

Future work will thus be targeted to: (1) extend the set of viewing features accessible through hand gesture (in

particular object selection and de-selection); (2) further improve the robustness of the system, particularly with

respect to different users; and (3) develop a larger system containing multiple cameras and enabling the interaction

of multiple users with different screens.

6. REFERENCES

Erol, A., Bebis, G., Nicolescu, M., Boyle, R. D., and Twombly, X. (2007). “Vision-based hand pose estimation: a

review.” Computer Vision and Image Understanding, vol. 108, 52-73.

27


Fox, A., Johanson, B., Hanrahan, P., and Winograd, T. (2000). “Integrating information appliances into an

interactive workspace.” IEEE Computer Graphics & Applications, vol. 20, no. 3, 54-65.

Gross, M., Würmlin, S., Naef, M., Lamboray, E., Spagno, C., Kunz, A., Koller-Meier, E., Svoboda, T., Van Gool,

L., Lang, S., Strehlke, K., Moere, AV., and Staadt, O. (2003). “Blue-c: a spatially immersive display and 3D

video portal for telepresence.” ACM Transactions on Graphics, 819-827.

Halatsch, J., and Kunze, A. (2007). “Value Lab: Collaboration In Space.” IV 2007: 11th International Conference

Information Visualization, Zurich, Switzerland, July 4-6, 376-381.

Halatsch, J., Kunze, A., Burkhard, R., and Schmitt, G. (2008a). “ETH Value Lab - A Framework For Managing

Large-Scale Urban Projects.” 7th China Urban Housing Conference, Chongqing, China, Sept. 26-27.

Halatsch, J., Kunze, A., and Schmitt, G. (2008b). “Using Shape Grammars for Master Planning.” DCC 2009: 3rd

Conference on Design Computing and Cognition, Atlanta, Sept. 21-26, 655-673.

Jedynak, B., Zheng, H., Daoudi, M., and Barret, D. (2002). “Maximum entropy models for skin detection.” ICVGIP

2002: 3rd Indian Conference on Computer Vision, Graphics and Image Processing, Ahmadabad, India, Dec.

16-18, 276-281.

König, W. A., Bieg, H.-J., Schmidt, T., and Reiterer, H. (2007). “Position-independent interaction for large

highresolution displays.” IHCI 2007: IADIS International Conference on Interfaces and Human Computer

Interaction, Lisbon, Portugal, July 6-8, 117-125.

Müller, P., Wonka, P., Haegler, S., Ulmer, A., and Van Gool, L. (2006). “Procedural Modeling of Buildings.” ACM

Transactions on Graphics, vol. 25, no. 3, 614-623.

Schmugge, S. J., Jayaram, S., Shin, M. C., and Tsap, L. V. (2007). “Objective evaluation of approaches of skin

detection using ROC analysis.” Computer Vision and Image Understanding, vol. 108, 41–51.

Van den Bergh, M., Koller-Meier, E., and Van Gool, L. (2009). “Real-time body pose recognition using 2D or 3D

Haarlets.” International Journal on Computer Vision, vol. 83, 72-84.

28


TOWARDS A COLLABORATIVE ENVIRONMENT FOR SIMULATION BASED DESIGN

Michele Fumarola, Ph.D. Candidate,

Systems Engineering Group

Faculty of Technology, Policy and Management, Delft University of Technology

Jaffalaan 5, 2628 BX Delft, The Netherlands;

Tel. +31 (0)15 27 89567

[email protected]

http://www.tudelft.nl/mfumarola

Stephan Lukosch, Assistant Professor,




Tel. +31 (0)15 27 83403

[email protected]

http://www.tudelft.nl/sglukosch

Mamadou Seck, Assistant Professor,




Tel. +31 (0)15 27 83709

[email protected]

http://www.tudelft.nl/mseck

Cornelis Versteegt, Senior Project Manager,

APM Terminals Management BV

Anna van Saksenlaan 71, 2593 HW The Hague, The Netherlands;

[email protected]

ABSTRACT: Designing complex systems is a collaborative process wherein modeling and simulation can be used

for support. Designing complex systems consists of several phases; specification, conceptual and detailed design

and evaluation. Modeling and simulation is currently mostly used in the evaluation phase. The goals, objectives and

IT support for each phase differ. Furthermore, multi-disciplinary teams are involved in the design process. We aim

at providing an integrated collaborative environment for modeling and simulation throughout entire design

projects. The proposed architecture, called Virtual Design Environment, consists of three main components: a

design, a visualization, and a simulation component. The layout of the design is made in the design component. The

design component has been developed as an AutoCAD plug-in. This approach was chosen, due to AutoCAD being

used in many complex design projects. The AutoCAD plug-in communicates the design decisions to the simulation

component. The processes that will take place once the system is built, are simulated by the simulation component.

Finally, the results of the simulation are sent to the visualization component. The visualization component provides

an interactive 3D environment of the design and can serve decision makers as a tool for communication, evaluation

and reflection. In this paper, we present the architecture of this environment and show some preliminary results.

KEYWORDS: Collaborative design, modeling and simulation, virtual environment, knowledge sharing, complex

systems

1. INTRODUCTION

As introduced by Simon (1977), decision making is composed of structuring the problem, evaluating alternatives

upon criteria and selecting the best alternative. Modeling and simulation (M&S) is often seen as a tool to analyze

29


(Zeigler & Praehofer, 2000) and is therefore mainly used in the evaluation phase of a decision making process. We

propose that M&S can be introduced much earlier in the design process of complex systems; more precisely in the

phase of structuring the problem. Many different design methodologies can be found in literature for the design of

complex systems (INCOSE, 2009; Pielage, 2005; Sage & Armstrong, 2000). There are many similarities in all of

these design methodologies. We do not focus on an individual design methodology, but more on the similar phases

in the methodologies. We identify the following phases; specification, conceptual and detailed design, and

evaluation. Simulation can add value in each of these mentioned phases. Currently, however, simulation is mostly

used in the evaluation phase. When using simulation earlier in the design, the designers and decision maker’s ability

to generate alternatives will be enhanced. The designers can generate more alternatives and study them more

comprehensively. However, designing a complex system is commonly a collaborative process wherein multiple

actors are involved. These actors have varying interests and fields of expertise. To achieve a fruitful process, these

actors must first acquire a shared understanding of the problem domain and afterwards be able to collaborate

effectively on this problem.

To design a complex system, support is therefore needed from different perspectives. An integrated environment to

support the design of a complex system should not only be able to support design, but also simulation. As a shared

understanding among all involved actors (Piirainen, Kolfschoten, & Lukosch, 2009) is a major challenge in

collaborative design, the design and the simulation results need to be visualized to present them to the various actors

active in the design process. Moreover, these actors need to able to collaborate on the design by simulating and

visualizing the result. These perspectives are shown in Figure 1. We therefore propose to use a collaborative design

environment, based on M&S which supports designers in all the design phases mentioned earlier.

FIG. 1 The design environment should cover different perspectives.

In this paper, we will present how such an environment can be achieved. We will begin by describing a case study

we conducted at a large container operator which is currently dealing with the design of automated container

terminals. Designing a container terminal is a complex problem and automation brings additional challenges as it is

a novel approach wherein little experience has been gathered so far. From this problem, we will gather requirements

which will be used to design the environment. After discussing the related work on this topic, we will present our

approach for such an environment. Subsequently, some preliminary experiences will be presented based on

interviews with domain experts. We will finally conclude the paper and present our future work.

2. REQUIREMENTS ANALYSIS

Starting from an exploratory case study performed at a container terminal operator, we identify requirements for the

design environment that we propose. We will first present the case study and focus on a common approach on

designing a complex system as for instance a container terminal. From there, we will perform an analysis and extract

general requirements.

30


2.1 Case study

Automated container terminals are gaining momentum as the advantages in terms of costs and productivity are

getting clear. In this novel type of terminals, operations (e.g. picking up a container, placing it on a vessel, etc) are

computer operated, which has a number of advantages: lower life cycle costs, improvement in terms of safety,

reduction of damage to containers, higher service levels, etc. Designing these terminals is however a complex

endeavor: a large number of actors are involved in the design process and the design space (i.e. the number of

alternative designs) is quite large. Moreover, experience in the design of automated container terminals is still

limited as the number of these terminals around the globe is small. During the design process, two types of situations

may occur: brainstorming sessions and reflection.

In the brainstorming sessions, the different people involved in the process, gather in the same location and work

together on the design of the new terminal. In these situations, a white board is mostly used to sketch the design and

to present ideas. Sketching the design is however mostly coarse, and often the design is quickly described in words

e.g. stating the number of needed equipment without specifying the location. Supporting documents are shared

between the participants, notes are usually taken on paper or typed on a computer, and there is verbal

communication.

In the reflection, the actors involved in the design, work separately to concentrate on their particular task: e.g. the

CAD designer will make an initial drawing of the terminal, and the business analysts will use their business model.

There is little interaction between these actors until they reach a certain goal for which a new meeting will be

scheduled.

2.2 Analysis

The brainstorming sessions are mainly paper based and unstructured. As certain expertise is needed to use CAD

environments such as AutoCAD, these tools are seldom used in such sessions. Although CAD designers are skilled

in these environments, the remaining actors are not. These environments do however offer the opportunity of

making exact initial designs without running into misunderstandings. The first requirement is therefore the

possibility of sharing a CAD environment without exposing non-experts with unneeded complexity. Thereby, non-

experts have the possibility to gain a better understanding of the current design which is one of the major challenges

in complex collaborative design projects (Piirainen, et al., 2009).

In these design, understanding the dynamics of the system can be a hard task. As the system comprises a large

amount of entities, the exploration of alternatives and the experimentation with these alternatives needs to be

supported. Inputting a decision in the design environment should therefore be facilitated by taking into account such

things as contrasting decisions, physical feasibility, future outcomes, etc. The link between the design environment

and a possible simulation environment is hereby made.

The decision making process is supported by a large amount of documentation which is mainly printed or shared

through a computer network. Querying for a specific item of information can be challenging and inefficient. This is

due to the lack of structure of the common approach of sharing documentation. Having the ability to easily find the

right document to make a decision is important. This is however only possible if the right information is available in

the decision making environment: having the possibility to easily share documentation is therefore required.

For enabling collaborative decision making, a collaborative environment has to support a number of functionalities.

At the core of a collaborative decision-making process, actors have to reach a shared understanding of the problem

domain. For that purpose, the actors need to communicate their understanding to the others actors and they need the

possibility to discuss the feasibility of their decisions. The latter also requires understanding of recent developments

and changes in the design. As a result, the collaborative environment has to offer various means for synchronous as

well as asynchronous communication and mechanisms for achieving group awareness (Gutwin, Greenberg,

Roseman, & Sasse, 1996).

From this analysis, a number of requirements can be extracted:

1. Visualization for non-experts of the system under investigation

2. Documentation sharing and structuring

31


3. Simulation-based experiments

4. Communication and awareness support for the participants

3. RELATED WORK

Modeling and simulation has been used often in the analysis of container terminal design. Stahlbock & Voss (2008)

reviewed the existing literature on container terminal logistics and found simulation as a tool to analyze different

aspects of container terminals. Nevertheless there is little reference to the use of M&S during the synthesis phase of

the design process. Ottjes et al. (2006) discuss their solution using three main functions to make the conceptual

design of a container terminal. This environment however, mainly focuses on simulation, leaving out the

collaborative aspects. More attention on collaboration in simulation projects for container terminal logistics have

been put by Versteegt, Vermeulen, & van Duin (2003), although existing tools were used which do not offer explicit

support for collaboration.

On the design of complex systems, more work has been done which comes closer to fulfilling our stated

requirements. Peak, et al. (2007) presented their approach based on SysML where they use components to design a

system, they do not however concentrate on collaboration during the design. Paredis et al. (2001) also introduced a

component based simulation environment for the design of mechatronic systems. Their solution enables multiple

users to collaborate on the design of such a system, but do not take into account the different skills of the actors

involved, as their scenario assumes users with comparable backgrounds. Comparable environments exist for virtual

prototyping in specialized engineering (e.g. automobile, aeronautical) industries.

On collaborative design in virtual environments, various examples are at hand. Shiratuddin & Breland (2008)

present an environment for architectural design that uses a 3D game engine to present the final design. They argue

shared understanding is achieved across interdisciplinary groups. Rosenman, Smith, Maher, Ding, & Marchant

(2007) discuss a solution that has different views (CAD and 3D virtual environment) on a given design in the AEC

domain. However, simulation is out of the scope of this research. Further examples of collaborative design in virtual

environments can be found in Conti, Ucelli, & Petric (2002), and Pappas, Karabatsou, Mavrikios, & Chryssolouris

(2006).

The conclusions found in Sinha, Lian, Paredis, & Khosla (2001) suggest a lack of close integration of design and

modeling & simulation tools. They also recommend the use of design repositories that would provide a way to share

knowledge about the system that is being designed. Furthermore, the few integrated environments do not support

collaboration across interdisciplinary groups.

4. APPROACH

Based on the requirements set in section 2, we will present our approach which comprises 4 parts: visualization,

sharing, simulation, and collaboration. These parts come forward from the different requirements. Firstly, we

discussed the need of non-experts to be able to understand the complex designs made in environments such as

AutoCAD. In order to do so, an understandable way of visualizing such a design is needed. Secondly, we identified

the requirement of sharing documents which are important for the decision making process. Thirdly,

experimentation with a given design is desirable, for which simulation is needed to predict the workings of the

system. Lastly, communication between the different actors needs to be supported as well as collaboration while

working on the design.. With these parts, an architecture can be finally constructed, which we will discuss as well.

4.1 Visualization

During the design process, visualization plays an important role in order to understand the problem at hand. In the

case of container terminals, it becomes even more important as the point of focus is a physical facility.

The design of a container terminal is commonly done in a CAD environment such as AutoCAD. This offers the

designers the possibility to precisely specify the design and later to use the drawings for the construction process.

CAD drawings are usually rather complex, making them hard to use by less proficient users such as business

analysts. An alternative way of visualizing the future facility is therefore required.

32


Various studies have been performed showing 3D as the preferred approach on visualizing complex physical

systems to increase interest and understanding of a user and to facilitate dialogue (Fabri, et al., 2008; Terrington,

Napier, Howard, Ford, & Hatton, 2008). It is therefore desirable to have a translation from the CAD drawings to a

3D environment understandable by the actors involved in the design process. According to Whyte et al. (2000) this

translation can be achieved in several ways: by the library approach, the simple translation and the database

approach. The library approach uses a set of predefined 3D models to map with the CAD drawing. On the other

hand, the simple translation purely transforms the CAD drawing to a 3D model, using CAD drawings which are

drawn with 3D vectors. Lastly, the database approach uses a central database with a description of an object from

which a 3D model and a CAD drawing can be extracted.

Once this translation took place, the visualization is in place to be used. The designers interact with the 3D

environment to input their design decisions and to support their communication on ideas and possible issues. This

relieves them from having to describe everything in words, as what happens traditionally.

4.2 Sharing

The design process of a complex system is commonly accompanied by a large amount of documentation. A lack of

structure and inability to look through this documentation slows down the design process. Furthermore, everyone

involved in the design process should have access to the available documentation. Because of this, sharing and

structuring documentation should be part of the design environment.

The structuring of documentation can exploit the structure of the actual system. An ontology of the system would

provide the entities to which the documentation can be linked. In the case of a container terminal, the structure of the

system would be the physical structure of the terminal: e.g. yard, stacks, and different types of equipment. If a user

would want to find out information about a specific entity, the logical place to look for would be the visualized

entity in the virtual environment.

Using the structure, sharing can also become a possibility. By providing the functionality of adding documentation

by choosing a specific entity, this can be achieved. Furthermore, additional information such as notes, ideas and

issues can be shared as well using the same functionality.

4.3 Simulation

When a system is composed of many parts interacting dynamically in a non-trivial way, it becomes difficult to

understand or predict their performance through mere static analysis. In this context, simulation is a powerful

instrument for enabling experimentation with the system-to-be, in order to anticipate the implications of the design

decisions at a relatively low cost. Simulation is routinely used in this way to evaluate design alternatives through

what-if analysis.

For most clients involved in a modeling and simulation effort, only the experimentation results (and their

interpretations) have added value. All preceding phases, although appraised as essential, are typically seen as

technical work and are naturally externalized. This situation stems from the fact that advanced modelling and

simulation requires specialized training and is not at the core of the client’s business. This state of affairs produces a

gap between the client and the constructor of the simulation. On the one hand, the constructor’s only input is a

system description document that could never capture the richness of the design activity. On the other hand, the

client loses the benefits that would have accrued from using simulation earlier in the cycle to explore a larger design

space.

A collaborative environment integrating design and simulation would bridge the proverbial gap, allowing the (CAD)

designs to be translated into simulations models by an underlying simulation environment and thus facilitating

experimentation within the design activity, fostering creativity and reactivity. There is of course no magic bullet.

Such capability will only be possible if a comprehensive library of domain specific models has been constituted and

individually validated beforehand. An ontology guarantying the compositional soundness of the design is also an

essential asset. This ontology can be developed using the System Entity Structure formalism (Zeigler & Hammonds,

2007). The use of simulation formalisms supporting modularity and hierarchical construction of models is a strong

requirement of the simulation part. A system theoretic formalism such as DEVS is particularly suited for this goal

because due to its property of closed under coupling.

33


4.4 Communication and awareness support

The major requirements for enabling collaborative decision-making are related to achieve a shared understanding by

means of communication and awareness support. According to Gerosa, Fuks, Raposo, & Lucena (2004) cooperating

users must be able to communicate and to coordinate themselves. When communicating, users might generate

commitments and define tasks that must be completed to accomplish the common group goal. These tasks must be

coordinated so that they are accomplished in the correct order and at the correct time with respect to possible

external restrictions. To accomplish these tasks the users have to cooperate in a shared environment. However, while

cooperating, unexpected situations might emerge that demand new communication. In such communication new

commitments and tasks might be defined, which again must be coordinated to be accomplished in cooperation. In

this cooperation cycle, awareness plays a central role. Every user action that is performed during communication,

coordination, or cooperation generates information. Some of this information involves two or even more users, and

should be made available to all cooperating users so that they can become aware of each other. This helps to mediate

further communication, coordination, and cooperation and build up a shared understanding of their common group

goals and to synchronize their cooperation.

Considering the above, we decided to link the different components of the system architecture by means of

communication functionality and awareness widgets. In the following, we briefly describe the functionality we want

to integrate by means of patterns1 for computer-mediated interaction (Schümmer & Lukosch, 2007) which capture

best practices in the design of collaborative environments. For awareness purposes, we want to integrate a USER

LIST, an INTERACTIVE USER INFO, a CHANGE INDICATOR, a TELEPOINTER, a REMOTE FIELD OF VISION. The USER

LIST will show which decision-makers are currently present and take a look a the design visualization. The

INTERACTIVE USER INFO will be coupled with the USER LIST allow decision-makers to directly select between

different communication possibilities when choosing one decision-maker from the USER LIST. We will use the

CHANGE INDICATOR pattern within the visualization environment to highlight changes which a decision-maker has

not seen yet and thereby make decision-makers aware of the recent changes of the designer. The TELEPOINTER will

allow the stakeholders in the visualization environment to point to specific design parts and thereby support an

ongoing discussion. Finally, the REMOTE FIELD OF VISION will allow decision-makers to identify in which parts of

the simulation the others are interested in. Thereby, REMOTE FIELD OF VISION will foster discussion among the

decision-makers as well as raise the level of shared understanding.

To further increase the shared understanding, we decided to add functionality for synchronous as well as

asynchronous communication. We want to integrate an EMBEDDED CHAT, a FORUM, SHARED ANNOTATIONS as well

as a FEEDBACK LOOP. The EMBEDDED CHAT will be available in the visualization environment. It will allow

decision-makers to directly communicate with each other and discuss general questions concerning the design. We

will also include a FORUM in which decision-makers can start asynchronous discussions. By using THREADED

DISCUSSIONS these FORUMS can also serve a knowledge base and repository for the VDE. To allow decision-makers

a artifact-centred discussion we will support SHARED ANNOTATIONS. Decision-makers will be able to add

annotations to specific points of interest within the visualization environment and share these annotations with the

other decision-makers. Furthermore, these SHARED ANNOTATIONS will be pushed to the design environment so that

the designer becomes aware of questions as well as discussions concerning the design of the complex environment.

Thereby, we implement a FEEDBACK LOOP. In addition, the designer will be able to comment on the annotations so

that a THREADED DISCUSSION can evolve within a SHARED ANNOTATION.

5. SOLUTION

5.1 Architecture

The different parts which result from the requirement, serve as a basis to construct a software architecture for the

proposed environment. Each part contains different components in order to achieve the required functionality. A

component diagram of this architecture is shown in Figure 2.

The visualization part contains a 3D visualizer and 2D textual output. The 3D visualizer renders the 3D environment

which is based on the CAD drawing. The visualization is currently being handled in a rendering engine, namely

1 Please note that pattern names are set in SMALL CAPS.

34


Ogre3D. This enables us to accurately render the large amount of objects commonly found in container terminals.

The 2D textual input provides the possibility to visualize textual information as well as pictures, videos, etc. This is

needed for documentation sharing and some functionality to enable collaboration.

The environment is fed by different sources. First of all, a database can feed the documentation to the environment

and information can also be added by users who wish to share it. On the other hand, simulation results can be used

as input by the simulation data feed.

The user interface gives access to the virtual environment with its documentation and simulation results. This is the

main interface for the decision makers involved in the design process. The world loader reads the CAD drawing

which is being edited by the designers. Once the designers finish to work on the layout, the CAD drawing is send to

the 3D virtual environment which will visualize the new design. This design will hereafter serve to run the

simulation and thereby study the performance of the new design. Feedback from non-designers can hereafter be fed

back to the CAD environment making it possible for the designers to handle new requests. The interaction between

designers and non-designers is supported as outlined in the previous section. This process can run multiple times

throughout the design process. The process is sketched in Figure 3.

5.2 Discussion

The environment presented here provides the means to collaboratively design a system using simulation across a

multidisciplinary group of actors. To achieve this, we presented an architecture based on the different parts resulting

from the requirements. This provides us an environment wherein a complete design process can take place. In

contrast to existing environments (for instance the ones discussed in the related work-section), the presented

environment provides the possibility to design a complex system collaboratively, visualize the design to achieve

shared understanding and support the design by sharing existing knowledge. The strength of the environment is

therefore not found in the individual components, which are already known and widespread, but in the integrated

environment wherein the components reside.

The design of the system is supported by simulation which is used to evaluate viable alternatives. Actors can achieve

insight into the workings of the systems which has not been physically developed yet. The actors, which have

different backgrounds, use a view on the system which they can understand (2D CAD or 3D) and can share existing

and new knowledge on the given system. Lastly, communication between these actors is possible through the

environment.

FIG. 2 An architecture for the proposed environment.

35


6. EXPERIENCES

Initial feedback from designers and decisions makers that have worked with the Virtual Design Environment is very

positive. First, the Virtual Design Environment provides more insight by using photorealistic 3D images and movies

of high quality. The designers used 2D images and movies in the past. The users especially appreciated the realism

of the environment. The Virtual Design Environment uses models and CAD drawing to create a realistic

representation. Schematic representations were used in the past. Secondly, the Virtual Design Environment is an

environment in which the users can freely move around. Traditionally, the movies contained predefined flight paths

that offered no flexibility in the point of view of the designers. The designers and decision makers indicated that

they see high value in using the Virtual Design Environment. They expect a number of benefits that were not

anticipated by the developers. The Virtual Design Environment can be used as a training tool before the terminal is

implemented. Operators can be trained for their future job using the Virtual Design Environment. The operators can

be trained in a safe environment without disrupting day-to-day operations. The decision makers also expect that the

Virtual Design Environment will have high value for the commercial aspects of the terminal. The Virtual Design

Environment can be shown to customers and used as a “selling” tool. Finally, the Virtual Design Environment can

be used for communicating the design to port authorities, governments and other stakeholders. The design can be

presented in an understandable format to all stakeholders.

7. CONCLUSIONS

In this paper we presented our work towards the Virtual Design Environment, an environment meant to support the

design of complex systems in a multi-actor setting. From an exploratory case study at a large container terminal

operator, we gathered the requirements for the design environment. These requirements were marked as five

components: visualization, sharing, communication, collaboration, and simulation. Although the design environment

is still under research, preliminary results could be gathered. Consultation with domain experts showed that the

design environment can indeed result in a more effective and efficient design process. Nevertheless, extended

evaluations have to be done to confirm these early expectations. Future work will therefore consist of further

development of the environment. Moreover, generalizing the environment should be considered, instead of

restraining it to the design of automated container terminal.

8. REFERENCES

Conti, G., Ucelli, G., & Petric, J. (2002). JCAD-VR: a collaborative design tool for architects. Paper presented at the

4th International Conference on Collaborative virtual environments.

Fabri, D., Falsetti, C., Iezzi, A., Ramazzotti, S., Rita Viola, S., & Leo, T. (2008). Virtual and Augmented Reality. In

H. H. Adelsberger, Kinshuk, J. M. Pawlowski & D. G. Sampson (Eds.), Handbook on Information

Technologies for Education and Training (pp. 113-132): Springer Berlin Heidelberg.

FIG. 3 Design process using the Virtual Design Environment

36


Gerosa, M. A., Fuks, H., Raposo, A. B., & Lucena, C. J. P. (2004). Awareness Support in The AulaNet Learning

Environment. Paper presented at the International Conference on Web-based Education.

Gutwin, C., Greenberg, S., Roseman, M., & Sasse, M. (1996). Workspace Awareness in Real-Time Distributed

Groupware: Framework, Widgets, and Evaluation. Paper presented at the People and Computers XI

(Proceedings of the HCI'96).

INCOSE (2009). The International Council on Systems Engineering Retrieved July 13th, 2009, from

http://www.incose.org

Ottjes, J. A., Veeke, H. P. M., Duinkerken, M. B., Rijsenbrij, J. C., & Lodewijks, G. (2006). Simulation of a

multiterminal system for container handling. OR Spectrum, 28(4), 447-468.

Pappas, M., Karabatsou, V., Mavrikios, D., & Chryssolouris, G. (2006). Development of a web-based collaboration

platform for manufacturing product and process design evaluation using virtual reality techniques.

International Journal of Computer Integrated Manufacturing, 19(8), 805-814.

Paredis, C. J. J., Diaz-Calderon, A., Sinha, R., & Khosla, P. K. (2001). Composable Models for Simulation-Based

Design. Engineering with Computers, 17(2), 112-128.

Peak, R. S., Burkhart, R. M., Friedenthal, S. A., Wilson, M. W., Bajaj, M., & Kim, I. (2007). Simulation-Based

Design Using SysML - Part 2: Celebrating Diversity by Example. Paper presented at the INCOSE

International Symposium.

Pielage, B.-J. (2005). Conceptual Design of Automated Freight Transport Systems: Methodology and Practice. Delft

University of Technology, Delft.

Piirainen, K., Kolfschoten, G., & Lukosch, S. (2009). Unraveling Challenges in Collaborative Design: A Literature

Study. Paper presented at the 15th Collaboration Researchers' International Workshop on Groupware.

Rosenman, M. A., Smith, G., Maher, M. L., Ding, L., & Marchant, D. (2007). Multidisciplinary collaborative design

in virtual environments. Automation in Construction, 16(1), 37-44.

Sage, A., & Armstrong, J. J. (2000). Introduction to systems engineering. New York, NY, USA: John Wiley & Sons

Inc.

Schümmer, T., & Lukosch, S. (2007). Patterns for Computer-Mediated Interaction: John Wiley & Sons.

Shiratuddin, M. F., & Breland, J. (2008). Development of a Collaborative Design Tool for Virtual Environment

(CDT-VE) Utilizing a 3D Game Engine. Paper presented at the 8th International Conference on Construction

Applications of Virtual Reality 2008.

Simon, H. (1977). The new science of management decision. Upper Saddle River, NJ, USA: Prentice Hall PTR.

Sinha, R., Lian, V. C., Paredis, C. J. J., & Khosla, P. K. (2001). Modeling and Simulation Methods for Design of

Engineering Systems. Journal of Computing and Information Science in Engineering, 1(1), 84-91.

Stahlbock, R., & Voss, S. (2008). Operations research at container terminals: a literature update. OR Spectrum,

30(1), 1-52.

Terrington, R., Napier, B., Howard, A., Ford, J., & Hatton, W. (2008). Why 3D? The Need For Solution Based

Modeling In A National Geoscience Organization. Paper presented at the 4th International Conference on

GIS in Geology and Earth Sciences.

Versteegt, C., Vermeulen, S., & van Duin, E. (2003). Joint Simulation Modeling to Support Strategic Decision-

Making Processes. Paper presented at the 15th European Simulation Symposium and Exhibition.

Whyte, J., Bouchlaghem, N., Thorpe, A., & McCaffer, R. (2000). From CAD to virtual reality: modelling

approaches, data exchange and interactive 3D building design tools Automation in Construction, 10(1), 43-

55.

Zeigler, B. P., & Hammonds, P. E. (2007). Modeling & Simulation-Based Data Engineering: Introducing

Pragmatics into Ontologies for Net-Centric Information Exchange: Academic Press.

37


Zeigler, B. P., & Praehofer, H. (2000). Theory of modeling and simulation: Academic Press.

38


EMPIRICAL STUDY FOR TESTING EFFECTS OF VR 3D SKETCHING ON DESIGNERS’ COGNITIVE ACTIVITIES

Farzad Pour Rahimian, Dr.,

Department of Architecture, Faculty of Design and Architecture, Universiti Putra Malaysia;

[email protected]

Rahinah Ibrahim, Associate Professor Dr.,

Department of Architecture, Faculty of Design and Architecture, Universiti Putra Malaysia;

[email protected]

ABSTRACT: To optimise the level of information integration of the critical conceptual architectural-engineering

design stage, designers need to employ more flexible and intuitive digital design tools during this phase. We studied

the feasiblity of using 3D sketching in VR in order to replace current non-intuitive Computer Aided Design (CAD)

tools that designers would rather not using during the conceptual architectural process. Using the capablities of

VR-based haptic devices, the proposed 3D sketching design interface relies on the sense of touch for simplifing

designing and integrating designers’ cognitions and actions in order to improve design creativity. Adopting a

cognitive approch to designing, the study compares the effectivenss of the proposed VR-based design interface with

common manual sketching design interfaces. For this purpose, we conducted a two-session expeiment which

comprises of design activites of three pairs of 5th year architecture students. In comparing the designers’ collective

cognitive and collaboratibve actions the study employs design protocol analisys research methodology. This study

evaluated the designers’ spatial cognition at four different levels: physical-action, perceptual-actions, functional-

actions, conceptual-actions. The results show that compared to the traditional design interfaces, the utilized VR-

based simple and tangible interface improved designers’ cognitive design activities. We claim that due to the

capability of reversing any undesired changes, 3D sketching design interface increases designers’ motivation and

courage for performing more cognitive activities than conventional approach. Increasing the occurrence frequency

of designers’ perceptual actions, the 3D sketching interface associated cognition with action and supported the

designers’ epistemic actions which are expected to increase design creativity. The rich graphical interface in 3D

sketching system has led to the occurrence of more ‘unexpected discoveries’ and ‘situative inventions’ that carried

both problem and solution spaces towards maturity. Moreover, the increment in the percentage of new physical

action has decreased the amount of unnecessary physical actions and possibility for shifting from pragmatic actions

towards epistemic actions. Results of this study can help the development of cutting-edge information technologies

in either design or education of architecture. They also can help in the creation of training programs for

professional graduates who are competent in multidisciplinary teamwork and equally competent in utilizing IT/ICT

in delivering their building projects within time and budget.

KEYWORDS: Conceptual Design, 3D Sketching, Multidisciplinary, Virtual Reality, Protocol Analysis.

1. INTRODUCTION

Early conceptual phases of the design process are characterized by fuzziness, coarse structures and elements, and a

trial-and-error process. Craft and Cairns (2006) mentioned that searching for form and shape is the designers’

principal goal. During these stages the chances of correcting errors are the highest and the use of low-expenditure

sketches and physical models is crucial. Cross (2007) believes that the thinking processes of the designer hinge

around the relationship between internal mental processes and their external expression and representation in

sketches. Cross (2007, p.33) is confident that the designer has to have a medium “which enables half formed ideas to

be expressed and to be reflected upon: to be considered, revised, developed, rejected and returned to.”

Using current CAD tools has bad effects on designers’ reasoning procedures and hampers their cognitive activities

(Ibrahim and Pour Rahimian In review). Existing literature (e.g. Bilda and Demirkan, 2003) have also shown that

due to some inherent characteristics of current CAD tools, designers are not quite successful when they are working

with such digital design tools during conceptual design phase. However, literature recommends to designers to

migrate from manual design tools to digital design systems in order to integrate the whole design process (Kwon et

al. 2005). Literature also highlights that the intangible and arduous user interface of current CAD systems are two

39


major issues which hamper designers’ creativity during conceptual design phase (Kwon et al. 2005). Consequently,

this paper proposes VR 3D sketching which is a simple and tangible VR-based design interface as an alternative

solution to replace ordinary CAD systems during conceptual design phase. Regenbrecht and Donath (1996) posit

that 3D sketching in VR can be used as instantaneous reflection and feedback of the design procedure in which the

user can act with the digital design support tool in a spontaneous, game-like, empirical manner. Consequently, VR

can offer the ideal interface for free artistic visualization and linking creative experimentation and accurate

manufacturing-oriented modelling (Fiorentino et al. 2002). (Regenbrecht and Donath 1996)

This paper reports a conducted empirical experiment for reaffirming efficiency of the proposed system in conceptual

architectural design phase. The paper presents the results of a comparison on design activities between a VR-based

simple and tangible interface and a traditional pen and paper sketching interface. It focuses on designers’ collective

cognitive activities when working on similar design tasks. Here the traditional sketching method is selected as a

baseline to be compared to a proposed 3D sketching design methodology. The purpose is to reveal the cognitive

impacts of the proposed design system. Five pairs of 5th year architecture students experienced with the traditional

design and CAD systems were selected as participants for this experiment. During the experiment, protocol analysis

methodology (Ericsson and Simon 1993; Lloyd et al. 1995; Schön 1983a) was selected as a research and data

acquisition method to explore the effects of the different media on designers’ spatial cognition.

2. VR 3D SKETCHING AND COGNITIVE APPROACH TO DESIGNING

Goldschmidt and Porter (2004) defined designing as a cognitive activity which entails the production of sequential

representations of a mental and physical artefact. Tversky (2005) believes that constructing the external or internal

representations, designers are engaged in spatial cognition process in which the representations serve as cognitive

aids to memory and information processing. Schön (1992) asserted that with execution of action and reflection, each

level of representation makes designers evolve in their interpretations and ideas for design solutions. Such cognitive

approach to designing considers design media as something beyond mere presentation tools. In this approach,

reflections which are caused by design media are expected to either stimulate or hamper designers’ creativity during

design reasoning.

2.1 Creativity

The term, creative, is usually used as a value of a design artefact (Kim and Maher 2008). Yet, according to Visser

(2004) in cognitive psychology discussions this is linked to design activity which also comprises particular

procedures that have the potential to produce creative artefacts. Cross and Dorst (1999) define the creative design

procedure as a sort of non-routine designing activities that usually is differentiated from the others by the appearance

of considerable events or unanticipated novel artefacts.

‘Situative-inventions’ is a more evolved model for measuring design creativity. According to Suwa et al. (2000),

situated-invention of new design requirements (S-invention) can be considered as a key for inventing a creative

artefact. Based on this model, when introducing the new constraints for design artefact, designers capture significant

parts of the design problem and go beyond a synthesis of solutions that suits the given requirements. On the other

hand, Cross and Dorst (1999) posit the modelling of the design creativity as a co-evolution of problem and goal

spaces. Co-evolutionary design is an approach to problem-solving (Kim and Maher 2008). In this approach the

design requirements and design artefacts are formed disjointedly while mutually affecting each other. Kim and

Maher (2008) believe that in this approach the changing of a problem causes some changes in the designer’s insight

of a problem situation.

‘Unexpected discoveries’ (Suwa et al. 2000) is a key for evaluating creative design process. They define it as

perceptual activities of articulating tacit design semantics into visuo-spatial forms in an unanticipated way for later

inspection. They found that the appearance of unexpected discoveries of visuo-spatial forms and S-invention are

strongly related to each other. As another approach, Suwa and Tversky (2001) in a constructive approach posit that

‘co-evolution’ of new conceptual semantics and ‘perceptual discoveries’ improve designers’ understandings of

external representations. In Gero and Damski’s (1997) opinion, constructive perceptions allow designers to change

their focus and to understand design problem in a different way in which re-interpretation may be stimulated so that

designers find the opportunity to be more creative. (Suwa and Tversky 2001)

40


2.2 Spatial cognition using tangible user interfaces (TUIs) in VR 3D sketching

Tangible user interfaces (TUIs) is a technology which comprises of digital information and physical objects to

virtually mimic an absolute environment. According to Kim and Maher (2008), as opposed to a simple time-

multiplexed technology which is used in ordinary input devices (e.g. a mouse), the main advantages of TUIs is the

space multiplexing input technology which is able to control various functions at different times. One instance of

such machinery is haptic technology. In computer science discussions, the term ‘haptic’ relates to the sense of touch.

In other words, this technology is a technology which unites the user to a digital system by simulating the sense of

touch and applying force-feedback, vibrations, and motions to the user (Basque Research 2007). We believe that this

physical interaction with real world is the quality that Stricker et al. (2001) describe as the technology which

augments our cognition and interaction in the physical world. Based on its capabilities we believe that haptic

technology provides an advanced TUI for designers.

As mentioned above, in haptic technology the sense of touch is not limited to a feeling and it facilitates a real-time

interactivity with virtual objects. According to Brewster (2001), haptic technology is a huge pace in VR area since it

allows users to utilize their touch sense to feel virtual objects. He argues that although touch is an extremely

powerful sense, it has so far been abandoned in the digital world. In this research we focused on the role of force-

feedback facilitated by SensAble Technology TUIs in forming a designer’s spatial cognition. (Fitzmaurice 1996)

Fitzmaurice (1996) relates the effects of such interfaces to the quality of motor activities. He uses definitions of

epistemic or pragmatic actions (Kirsh and Maglio 1994) to classify designers’ motor activities. In his definition,

epistemic actions are taken to reveal hidden information or that are difficult for mankind to compute mentally. He

believes that the physical activities help people perform easier, faster and more reliable on internal cognitive

computation. This is something like using the fingers when counting. According to Fitzmaurice (1996) the epistemic

actions can improve cognition by: 1) decreasing the involvement of memory in mental computation (space

complexity), 2) decreasing the number of mental computation steps (time complexity), and 3) decreasing the rate of

mental computation error (unreliability). On the other hand, Fitzmaurice (1996) define pragmatic actions as physical

actions which primary physically perform to make the user closer to the aim.

Fitzmaurice (1996) believes that such activities can strongly help an integrated human cognitive model in which

necessary information for each step can be provided by both mental processing resources and physical modifying.

He argues that consequently this can support perceiving external environment. Fitzmaurice (1996) concludes these

arguments positing that mental modules can trigger motor activity which propose to lead to changes in the physical

environment that assists cognitive processes. He argues that when using physical objects, users are able to

manipulate and influence their environment. This paper seeks the effects of proposed VR 3D sketching interface of

designers’ collective cognitive activities based on above mentioned cognitive approach to designing.

3. COMPARING VR 3D SKETCHING AND TRADITIONAL SKETCHING

This section presents the developed methodological framework besides the details of the conducted experimental

protocol analysis for testing proposed VR 3D sketching interface. It comprises explanations of five-step conducted

protocol analysis methodology which is proposed by van Someren et al. (1994). The five steps are as follow: 1)

conducting experiments, 2) transcribing protocols, 3) segmentation procedure, 4) developing coding scheme and

encoding protocol data besides developing research hypotheses, and 5) selecting strategies to analyze and interpret

the encoded protocols. The section ends with the explanation of the adopted strategies for validation and reliability.

3.1 Experimental protocol analysis for testing VR 3D sketching design interface

As discussed above, this study proposes a simple and tangible VR-based design system as an alternative solution to

replace ordinary CAD systems during conceptual design phase. This section reports an empirical experiment for

reaffirming efficiency of the proposed system in conceptual architectural design phase. The study proposes a

comparison on design activities between a VR-based simple and tangible interface and a traditional pen and paper

sketching interface. It focuses on designers’ collective cognitive activities when working on similar design tasks.

Here the traditional sketching method is selected as a baseline to be compared to a proposed 3D sketching design

methodology. The purpose is to reveal the cognitive impacts of the proposed design system. Five pairs of 5th year

architecture students experienced with the traditional design and CAD systems were selected as participants for this

41


experiment. Each pair was required to complete two design tasks which utilized traditional and 3D sketching design

media sequentially. During the experiment, protocol analysis methodology (Ericsson and Simon 1993; Lloyd et al.

1995; Schön 1983a) was selected as a research and data acquisition method to explore the effects of the different

media on designers’ spatial cognition.

3.1.1 Development of research instrument, SensAble haptic devices vs. manual sketching

In order to compare the impacts of the proposed 3D sketching design methodology on the designers’ cognitive

activities, we proposed a simple traditional conceptual design package as a baseline system and a VR-based digital

design package as a 3D sketching environment. The utilized traditional conceptual design package comprises design

pencils and pens, butter papers, and simple mock-up materials e.g. polystyrene as well as drafting tables and chairs.

On the other hand, the proposed VR-based digital design package consists of a tablet PC for supporting designers’

preliminary ideations and a desktop PC for supporting digital design process. Both systems are shown in Figure 1.

FIG.1: Prepared traditional (left) and 3D sketching (right) design settings

The commercial software Adobe PhotoshopTM was installed on the tablet PC to provide the layering ability that was

available in the traditional sketching system which used butter papers. Designers therefore were able to produce

preliminary sketches directly on the screen of the tablet PC. The utilized desktop PC comprised of a monitor as the

output system and a keyboard, a mouse, and a 6DF SensAble haptic device which supported force-feedback and

vibration as the input system. During the experiment we used an evaluation version of ClayToolsTM software as the

basic environment for the modelling and spatial reasoning. ClayToolsTM is VR-based software which is designed for

being used with SensAble haptic devices. The experiment expects the used 6DF coordination system to solve the

previous 2D mouse coordination problems which designers faced in the traditional CAD systems. Ultimately, the 3D

sketching interface was set up to offer the expected simplicity and tangibility of using VR in design.

3.1.2 Design tasks

In order to test the effects of the interface on all aspects of conceptual design, the designers were required to perform

in two comprehensive conceptual design sessions for full three hours each. Therefore, during these sessions,

designers were asked to undergo all stages of conceptual design: initial bubble diagramming, developing design

idea, and preparing initial drawings. The goal of the first design task was to design a shopping centre with maximum

200000 square feet built area. On the other hand, the goal of the second design task was to design a culture and art

centre with maximum 150000 square feet built area. In order to make designers concentrate on design itself rather

than presentation, during both sessions they were required not to use more than one colour in the presentations.

3.1.3 Experimental set-ups: traditional session vs. 3D sketching session

We started our experiment with 5 pairs of designers. However, since 2 groups failed to complete their training

sessions, we performed the experiment for only 3 groups of designers. The traditional sessions were held at a design

studio while the 3D sketching sessions were held in an office which was being used as a VR lab during the

experiment. During both sessions to record all of the events during the design sessions, two digital cameras and one

sound recorder were used. The purpose of the first camera was to record all the drawings which were produced

during the test. The other camera was set up to record the designers’ design gestures and behaviours. Finally, a

digital sound recorder was used to record the designers’ conversations for transcription purposes. The designers

were asked to sit on one side of the table which was facing both cameras. Without interfering with the designers’

42


thinking process, the experimenter was present at the design studio and Lab to prevent any technical problem. The

explained setting is shown in Figure 2 and Figure 3.

FIG.2: Experimental set-up for traditional sessions

FIG.3: Experimental set-up for 3D sketching sessions (Ericsson and Simon 1993)

3.2 Protocol analysis

Due to a tendency towards the objective ways for studying designers’ problem-solving processes, protocol analysis

is the emerging prevailing method for design studies (Cross et al. 1996). Kim and Maher (2008) have advocated

using this methodology for analyzing and measuring designers' cognitive actions instead of using subjective self-

reports such as questionnaires and comments. Having all strategies of protocol analysis methodology, since our

study focuses on designers’ cognitive activities and also the use of concurrent method is impossible for collaborative

works (Kan 2008), the retrospective content-oriented protocol analysis is selected as the data collection strategy for

our research. Herewith, the designers worked naturally while the entire processes were recorded. After finishing

their sessions, the designers were required to transcribe their sessions using the aid of the recorded media. Their

transcriptions as well as the recorded media provided the research data.

3.2.1 Unit of analysis and strategy in parsing the segments

Since this research focuses on designers’ cognitive actions during both traditional and digital sessions and the tested

hypotheses are relying on their actions, the codes assigned to the different segments are considered as our units of

analysis. Besides, we followed Ericsson and Simon’s (1993) suggestions in segmenting the process depending on

the occurrences of the processes.

3.2.2 Coding Scheme

Basically, the coding scheme which is used in this study is borrowed and adopted from Suwa et al.’s (1998, 2000)

studies. The developed coding scheme comprises main categories from the Suwa et al.’s (1998, 2000) coding

scheme and our own sub-categories based on the designers’ different actions which were particular for our study.

Ultimately, the proposed coding scheme characterizes designers’ spatial cognition at four different levels: physical-

action, perceptual-actions, functional-actions, and conceptual-actions. Although it is not claimed that our sub-

categories are the best possible answers for this kind of study, we are confident that this coding scheme is capable to

embrace all cognitive codes that our designers produced during the experiment. Details are shown in Table 1.

43


TABLE. 1: Developed coding scheme for 4 action-categories and their sub-categories Category ID Index Description

Physical P ‐ Directly related to the P‐actions

D‐actions Da ‐ Depicting actions which create or deal with any visual external representation of design

CreateNew Dacn New To create a new design element or a symbol (drawing circles, lines, textures, arrows, etc)

ModifyExisting Dame New To edit the shape, size, texture etc of the depicted element

CreateMask Dacm Old To create a mask area for selecting something

RelocateExisting Dare New To change the location or the orientation of the depicted element

CopyExisting Dace New To duplicate an existing element (for digital work only)

TracingExisting Date Old To trace over the existing drawing

RemoVeExisting Dave New To remove an existing object or (for digital work only) to undo any command or to turn off

L‐actions La ‐ Look actions which include inspecting a previous depictions or any given information

InspectBrief Laib Old Referring to the design brief

TurnonObject Lato Old Turning on the invisible objects

InspectScreen Lais Old Looking at screen (for digital work only)

Inspect‐sHeet Laih Old Looking at design sheet (for manual work only)

Inspect3DModel Lai3 Old Looking at virtual or physical 3D model while rotating it

M‐actions Ma ‐ Other P‐actions which can fall into the motor activities

MovePen Mamp New To move pen on the paper or board without drawing any thing

MoveElement Mame New To move an element in the space arbitrarily for finding new spatial relationship

TouchModel Matm New To touch either physical or virtual model to stimulate motor activities

ThinkingGesture Matg New Any arbitrarily gesture which motivates thinking about design

Perceptual Pe ‐ Actions related to the paying attention to the visuo‐spatial features of designed elements or

space

P‐visual Pv ‐ Discovery of visual features (geometrical or physical attributes) of the objects and the spaces

NewVisual Pnv Unexp.D New attention to a physical attributes of an existing object or a space (shape, size or texture)

EditVisual Pev Other Editing or overdrawing of an element to define a new physical attribute

NewLocation Pnl Unexp.D New attention to the location of an element or a space

EditLocation Pel Other Editing or overdrawing of the location of an element or a space to define a new physical

attribute

P‐relation Pr ‐ Discovery of spatial or organizational relations among objects or spaces

NewRelation Pnr Unexp.D New attention to a spatial or organizational relations among objects or spaces

EditRelation Per Other Editing or overdrawing of a spatial or organizational relations among objects or spaces

P‐implicit Pi ‐ Discovery of implicit spaces existing in between objects or spaces

NewImplicit Pni Unexp.D Creating a new space or object in between the existing objects

EditImplicit Pei Other Editing the implicit space or object in between the existing objects by editing or relocating the

objects

Functional F ‐ Associating visual or spatial attributes or relations of the elements or the spaces with

meanings, etc

F‐interactions Fi ‐ Interactions between designed elements or spaces and people

NewInteractive Fni ‐ Associating a interactive function with a just created element or space or a spatial relation

ExistingInteractive Fei ‐ Associating a interactive function with an existing element or space or a spatial relation

ConsiderationInteractive Fci ‐ Thinking of an interactive function to be implemented independently of visual features in the

scene

F‐psychological Fp ‐ People’s psychophysical or psychological interactions with designed elements or spaces

NewPsychological Fnp ‐ Associating a psychological function with a just created element or space or a spatial relation

ExistingPsychological Fep ‐ Associating a psychological function with an existing element or space or a spatial relation

ConsiderationPsychological Fcp ‐ Thinking of an psychological function to be implemented independently of visual features

Conceptual C ‐ Cognitive actions which are not directly caused a visuo‐spatial features

Co‐evolution Ce ‐ Preferential (like‐dislike) or aesthetical (beautiful‐ugly) assessment of the P‐actions or F‐actions

Set‐up Goal activities Cg ‐ Abstracted issues out of particular situations in design representation which are general

enough to be accepted via the design process thoroughly as a major design necessity

GoalBrief Cgb Other Goals based on the requirements of the design brief

GoalExplicit Cge S‐inv Goals introduce by the explicit knowledge or previous cases

GoalPast Cgp S‐inv Coming out through past goals

GoalTacit Cgt S‐inv The goals that are not supported by explicit knowledge, given requirements, or previous goals

GoalConflict Chc S‐inv Goals devised to overcome problems which are caused by previous goals

GoalReapply Cgr Other Goals to apply already introduced functions in the new situation

GoalRepeateD Cgd Other Goals repeated through segments

3.2.3 Measurement of design protocols and testing hypotheses

After performing segmentation process and developing the coding scheme, based on recorded videos and transcribed

media we assigned related codes to every segment. Protocol analysis and interpretation starts only after assigning

related codes to every segment. In this study we relied on both descriptive and inferential statistics to analyze and

interpret the collected data. Graphs and charts have been employed in descriptive statistics to explore the meaningful

44


patterns of changes in designers’ cognitive actions. On the other hand, the inferential statistics has been employed

for testing the assumed hypotheses. For comparison purpose we used Wilcoxon ranks test which is equivalent to

paired sample t-test for non-parametrically distributed data. Then, for the purpose of testing relationship Chi-square

test have been selected for the cases in which the independent variables are categorical (nominal) and the dependent

variables are non-parametrically distributed ratio scaled values. For interpretation of the results, we compared the

test statistics against the significant value which is at (.05) for social sciences.

4. RESULTS AND ANALYSIS OF COLLECTED EMPIRICAL PROTOCOL DATA

This section presents the results and analysis data collected during the empirical study which was explained in

Section 3. The analysis in this chapter mostly hinges around four different levels of designers’ spatial cognition:

physical-action (P), perceptual-actions (Pe), functional-actions, and conceptual-actions (FC).

4.1 Overview of the coded data

Table 2 shows the mean value and standard deviation of segment duration for six sessions by three designer groups

as well as average value of each design method. For maintaining homogeneity of the results we tried to balance the

total time which every group spent for each of the sessions. The conducted Wilcoxon ranks test on coded data show

significant reduction in the average length of designers’ utterances during 3D sketching process compared to those

in traditional sketching process. According to Kim and Maher (2008) this could be considered as a good

phenomenon since it has potential for decreasing the load of designers’ mental cognitive processes.

TABLE. 2: Duration of segments for both traditional (Man) and 3D sketching (Digi) sessions

Session Total time (s) Segment num. Mean (s)/Std. Z value/Sig.

Pairs’ Ave. (Man) 10148 1941 14.61/20.67

Pairs’ Ave. (Digi) 10422 2581 10.78/14.80 8.77(a)***/.000

4.2 Analysis of designers’ spatial cognition

Table 3 exposes the occurrence frequency percentage of Physical-actions (P-actions), Perceptual-actions (Pe-

actions) and Functional-Conceptual actions during both manual and 3D sketching design sessions. The analysis and

of occurrence percentage of all three mentioned action categories are presented in related following sections.

TABLE. 3: Occurrence frequency percentage of cognitive activities (CA) for Manual and 3D Digital design sessions

Cognitive Activities

P-actions Pe-actions FC-actions Total

Manual % within the whole CA 74.9% 11.6% 13.5% 100.0% Mode CA

Digital % within the whole CA 69.1% 11.8% 19.1% 100.0%

4.2.1 Physical-actions

So far, based on the results coming from the selected sample we have one confirmed hypothesis which infers that

the proposed 3D sketching methodology can increase the total amount of the designers’ external cognitive activities

compared to the traditional design tools. For interpretation of this finding, the study needs to test the percentage of

the total P-actions among the whole cognitive actions. Table 3 shows decrement for percentage of the P-actions in

3D sketching sessions (69.1%) in comparison to those in traditional design sessions (74.9%). Conducted Chi-square

test confirms the assumed significance (X2=12.851, df=1, r<.001). As a consequence, the second hypotheses could

be confirmed. Based on results from the selected sample it can be inferred that compared to the traditional design

tools, the proposed 3D sketching methodology can decrease the occurrence frequency percentage of P-actions

among the entire designers’ external cognitive activities.

4.2.2 Perceptual-actions

In our coding system two major concerns are related to Pe-actions. The first concern is with the absolute number and

percentage of occurrence frequency of whole Pe-actions. The other concern is related to the occurrence of the

unexpected discoveries codes compared to the occurrence of the other perceptual codes. Figure 4 illustrates the

occurrence frequency percentage of the designers’ unexpected discoveries and the other Pe-actions during both

sessions. Results show that in all 3D sketching sessions there is an obvious increment (X2=9.889, df=1, r<.01) in

45


percentage of unexpected discoveries compared to the other Pe-actions. Moreover, according to the conducted

Wilcoxon ranks test to compare total amount of perceptual activities within similar allotted time, when using 3D

sketching methodology designers performed significantly more Pe-actions (Z=-6.489, p<.001) compared to what

they did during the traditional design sessions. Finally, in terms of visual analysis of the processes, referring to the

occurrence frequency scatter bars of Pe-actions reveals that in the two of the three cases occurrence of Pe-actions are

more consistent throughout the 3D sketching sessions compared to those in traditional sessions.

FIG. 4: Occurrence frequency percentage of the designers’ unexpected discoveries (Unex. Disc.), and the other Pe-

actions

4.2.3 FC-actions

In this section designers’ cognitive activities are categorized into three major categories. The first part comprises

designers’ all evaluations of their previous physical, perceptual, and functional actions. The second category belongs

to designers’ all functional and set-up goal activities. In this section, regardless of whether those actions are about

assigned functions or their set-up goals, all the codes are analyzed under the following two groups: 1) situative-

inventions, and 2) other FC-actions. Figure 5 illustrates the occurrence frequency percentage of all three groups of

the codes for all six traditional and 3D sketching design sessions. From the bar charts it can be concluded that

although the occurrence tendency of situative-inventions codes is almost the same (X2=1.509, df=1, r>.05) for all the

traditional and 3D sketching sessions, there is a huge increment for occurrence tendency of co-evolutions codes

during 3D sketching sessions compared to those in traditional session (X2=53.555, df=1, r<.001). This increment can

be considered as a consequent of the more explicit representations during 3D sketching sessions. Moreover, the

conducted chi-square test shows that there is a significant increment for occurrence percentage of FC-actions among

total cognitive activities (X2=17.179, df=1, r<.001). Finally, in analysing the processes visually, the occurrence

frequency scatter bars of FC-actions reveal that occurrence of FC-actions is more consistent throughout the 3D

sketching sessions compared to those in traditional sessions.

FIG. 5: Occurrence frequency percentage of the designers’ co-evolutions (Co-evol.), situative-inventions (S-inv),

and the other functional-conceptual (FC) actions during all six traditional (M) and 3D sketching (D) sessions

5. CONCLUSIONS

The main aim of this experiment was to provide objective and empirical evidence for the proposed VR-based 3D

sketching interface improves the designers’ spatial cognition during conceptual architectural design phase. In this

46


experiment the focus was on designers’ cognitive actions and the hypotheses were being tested relying on the

designers’ actions. The codes assigned to the different segments were considered as our units of analysis. Although

this experiment was made up of three pairs of designers performing six design sessions in total, the experiment

provides adequate data for observing overall designingly trends and actions. Besides, we were guided by Clayton et

al.’s (1998) recommendations in validating our results. Moreover, during our exploratory study we had observed

consistent improvements in the main five aspects of design sessions and spatial cognition across the three pairs that

further validated the claim that 3D sketching interface facilitates better quality of designing.

The study found that in 3D sketching sessions, the increased integration of the physical actions with mental

perceptions and conceptions would lead to occurrence of epistemic actions which improves the designers’ spatial

cognition. The results support (Kirsh and Maglio 1994) argument that the epistemic actions facilitated by the rich

interface would offload the designers’ mental cognition partly into the physical world, hence allowing them freer

mind to create more design ideas. Moreover, the 3D sketching interface improves the designers’ perception of visuo-

spatial features, particularly in terms of unexpectedly discoveries of spatial features and relationships. The

phenomenon we observed is explained by Schön (1983a) whereby there exist an association between mental

cognition and the perception of physical attributes that could stimulate creativity and offload the mental load.

Furthermore, the results support from Suwa et al.’s (2000) arguments to explain how unexpected discoveries can

lead to more creativity and also to the occurrence of more situative inventions.

In terms of functional-conceptual actions of the design process, we posit that 3D sketching interface would improve

the designers’ problem finding behaviours as well as improving their co-evolutionary conceptions of their

perceptions and problem findings. Suwa et al.’s (2000) explain these behaviours as ‘situative-inventions’ and argue

how the increased percentage of the co-evolutionary and situative-inventions actions can lead towards improved

creativity in 3D sketching design session. In conclusion, we argue that the emerging VR technologies are capable to

facilitate physical senses beyond the visual aspects of the design artefact by offering a new generation of promising

CAD tools which are constantly in touch with designers’ cognition during conceptual architectural design process.

6. ACKNOWLEDGMENTS

We acknowledge that this research is a part of doctoral study by the first author at Universiti Putra Malaysia (UPM)

which is partly sponsored by UPM's Graduate Research Fellowship (GRF). We also would like to gratefully

acknowledge contributions of the fifth year architectural students in the Semester 2 2008/2009 at the Faculty of

Design and Architecture, UPM. We also acknowledge the contributions of Prof. Dr. Mohd Saleh B. Hj Jaafar,

Associate Prof. Dr. Rahmita Wirza Binti O. K. Rahmat, and Dr. Muhamad Taufik B Abdullah during this study.

7. REFERENCES

Basque Research. (2007). "Using Computerized Sense Of Touch Over Long Distances: Haptics For Industrial

Applications." ScienceDaily.

Bilda, Z., and Demirkan, H. (2003). "An insight on designers' sketching activities in traditional versus digital

media." Design Studies, 24(1), 27-50.

Brewster, S. (2001). "The Impact of Haptic ‘Touching’ Technology on Cultural Applications." Glasgow Interactive

Systems Group, Department of Computing Science, University of Glasgow, Glasgow.

Craft, B., and Cairns, P. "Work interaction design: Designing for human work." IFIP TC 13.6 WG conference:

Designing for human work, Madeira, 103-122.

Cross, N. (2007). Designerly Ways of Knowing (paperback edition), Birkhäuser, Basel, Switzerland.

Cross, N., Christiaans, H., and Dorst, K. (1996). Analysing Design Activity, NY: Wiley & Sons, New York.

Cross, N., and Dorst, K. (1999). "Co-evolution of problem and solution space in creative design " Computational

models of creative design, J. S. Gero and M. L. Maher, eds., Key Centre of Design Computing, University of

Sydney, Sydney, 243-262.

Ericsson, K. A., and Simon, H. A. (1993). Protocol analysis: verbal reports as data, MIT Press, Cambridge

Fitzmaurice, G. W. (1996). "Graspable user interfaces," University of Toronto Toronto.

47


Gero, J. S., and Damski, J. (1997). "A symbolic model for shape emergence." Environment and Planning B:

Planning and Design, 24, 509-526.

Goldschmidt, G., and Porter, W. L. (2004). Design representation, Springer, New York.

Ibrahim, R., and Pour Rahimian, F. (In review). "Empirical comparison on design synthesis strategies by

architectural students when using traditional cad and manual sketching tools." Automation in Construction,

Special Issue for CONVR08's Selected Papers.

Kan, W. T. (2008). "Quantitative Methods for Studying Design Protocols," The University of Sydney, Sydney.

Kim, M. J., and Maher, M. L. (2008). "The impact of tangible user interfaces on spatial cognition during

collaborative design." Design Studies, 29(3), 222-253.

Kirsh, D., and Maglio, P. (1994). "On distinguishing epistemic from pragmatic action." Cognitive Science, 18(4),

513-549.

Kwon, J., Choi, H., Lee, J., and Chai, Y. " Free-Hand Stroke Based NURBS Surface for Sketching and Deforming

3D Contents." PCM 2005, Part I, LNCS 3767, 315 – 326.

Lloyd, P., Lawson, B., and Scott, P. (1995). "Can concurrent verbalization reveal design cognition?" Design Studies,

16(2), 237-259.

Regenbrecht, H., and Donath, D. (1996). "Architectural Education and VRAD." uni-weimar.

Schön, D. (1983a). The Reflective Practitioner: How Professionals Think in Action, Temple Smith, London.

Schön, D. (1992). "Designing as reflective conversation with the materials of a design situation " Knowledge-Based

Systems, 5(1), 3-14.

Stricker, D., Klinker, G., and Reiners, D. (2001). "Augmented reality for exterior construction applications."

Augmented reality and wearable computers, W. Barfield and T. Caudell, eds., Lawrence Erlbaum Press, 53.

Suwa, M., Gero, J. S., and Purcell, A. T. (2000). "Unexpected discoveries and S-inventions of design requirements:

important vehicles for a design process " Design Studies, 21(6), 539-567.

Suwa, M., and Tversky, B. (2001). "How do designers shift their focus of attention in their own sketches? ."

Diagrammatic reasoning and representation, M. Anderson, B. Meyer, and P. Olivier, eds., Springer, Berlin,

241-260.

Tversky, B. (2005). "Functional significance of visuospatial representations." Handbook of higher-level visuospatial

thinking, P. Shah and A. Miyake, eds., Cambridge University Press, Cambridge, 1-34.

van Someren, M. W., Barnard, Y. F., and Sandberg, J. A. C. (1994). The Think Aloud Method: A Practical Guide to

Modelling Cognitive Processes, Academic Press, London.

Visser, W. (2004). "Dynamic aspects of design cognition: elements for a cognitive model of design." Theme 3A-

Databases, Knowledge Bases and Cognitive Systems, Projet EIFFEL, France.

48


ANALYSIS OF DISPLAY LUMINANCE FOR OUTDOOR AND MULTI-USER USE

Tomohiro Fukuda, Associate Professor,

Graduate School of Engineering, Osaka University;

[email protected] and http://y-f-lab.jp/fukudablog/

ABSTRACT: The use of digital tools outdoors is anticipated, but a problem exists that when an ordinary PC display

is used it is hard to see because of outside light. To clarify the cause, three elements of the display were evaluated,

namely luminance, the contrast ratio, and the viewing angle. Five displays were assessed by using a luminance

meter, and the three elements of each display were measured in a darkroom. To decrease the various factors

affecting luminance measurement outdoors, the illuminance change outdoors, the influence of sunlight, and the

influence of the ambient surroundings were considered. Also, using experimental methodology that reduced these

factors, data about the three elements were acquired outdoors. Future work will need to clarify the element of the

luminance of the display outdoors.

KEYWORDS: Display for outdoor, luminance, contrast ratio, viewing angle, digital tool.

1. INTRODUCTION

Digital tools for multi-media purposes including MR�Mixed Reality�are expected to be used outdoors by many

users. However, in general, the medium of paper is used in design studies and by tour guides due to the various

problems that digital tools have (left and middle of FIG.1). In this study, from the various problems affecting digital

tools, the problem of the display is targeted. In general, the display is not easy to see outdoors due to the influence of

the outside light (Right of FIG.1).

FIG. 1: Sharing information outdoors using paper (left and middle); State of display outdoors (right).

Digital tools used outside by individuals, such as cellular phones, PDAs or HMDs (Head Mounted Display), were

not the target of this study. Instead the focus was on digital tools with which a number of people can share

information. Many papers have reported on digital tools used outdoors (Feiner, 1997; Behringer 2000; Julier 2000;

Baillot 2001; Kuo, 2004; Onohara, 2005). However, those papers described the feature development of the digital

tools, and did not consider the problem of ease of viewing the display outdoors.

The author has developed an MR system which includes a video image displaying a present image, and a real-time

3DCG image displaying images of objects or scenes that do not exist, such as design proposals or demolished

buildings in real time (Fukuda, 2006; Kaga, 2007). The set-up of this system for outdoor, multi-user, and mobile use

includes a tablet PC, MR software, a live camera, RTK-GPS (Real Time Kinematic - Global Positioning System),

and a 3D motion sensor. This system is expected to be used for city tours or design studies in the areas of education,

research, and practice.

In the presented paper (Fukuda, 2009), a problem of the developed MR system was shown, namely that the display

is hard to see outdoors because of the influence of the outside light. To grasp the problem quantitatively, three

displays were measured. However, this was a preliminary study and further research on the conditions of display

experiment used outdoors is needed.

49


This study aims to examine the problem of the display being hard to see outdoors. Two displays in addition to those

described in the presented paper (Fukuda, 2009) were added to improve the research method. In addition, the

experiment was carried out on five displays after the problem and measurements of the experiment on outdoor were

clarified.

2. DISPLAY EVALUATION ELEMENT

The brightness of a display is expressed with a luminance and a contrast ratio. Luminance is a photometric measure

of the luminous intensity per unit area of light travelling in a given direction. Where F is the luminous flux or

luminous power (lm), �is the angle between the surface normal and the specified direction, A is the area of the

surface (m2), and � is the solid angle (sr), Luminance L (cd/m2) is defined by

(1)

The contrast ratio is the luminosity ratio of the maximum luminosity Lmax {(R, G, B) = (255, 255, 255)} and the

minimum luminosity Lmin {(R, G, B) = (0, 0, 0)} on a display. The higher the contrast ratio, the greater the difference

between Lmax and Lmin is. To raise the contrast ratio, Lmax is enlarged or Lmin is unlimitedly brought close to 0.

Contrast ratio CR is defined by

(2)

In addition, a viewing angle that assumes about ten people can see the display is set. The viewing angle is the

luminosity ratio of the display front luminosity Lf and the diagonal 45 display degree luminosity Ls. The lower the

value, the smaller the difference between the luminance from the front and the diagonal luminance. That is, the

screen can be seen very well from a diagonal viewpoint. Viewing angle VA is defined by

(3)

3. EXPERIMENT AND RESULT

3.1 Whole image of experiment

To grasp the characteristics of the display for outdoor use, luminance L was measured in the darkroom and outdoors.

The contrast ratio CR and the viewing angle VA were calculated based on the measured luminosity value. Display

makers usually provide the dark room contrast ratio, which is a contrast ratio usually measured in a dark room where

illuminance is 0. The dark room contrast ratio is the standard numerical value. However, since it is significantly

influenced by outdoor daylight, just the dark room contrast ratio of evaluation of an outdoor display is inadequate.

Therefore, it is necessary to measure the ambient contrast ratio, which is a contrast ratio that adds and measures the

conditions of fixed outdoor daylight. The luminance meter used was an LS-100 by Konica Minolta Sensing, inc. and

the illuminance meter used was a T-10 by Konica Minolta Sensing, inc.

The procedure used in the experiment is shown below:

• The display and the luminance meter were set up in the darkroom as shown in FIG. 2.

• The power supply on the display was switched off.

• Luminance at the center of the screen was measured five times.

• The power supply on the display was switched on.

• A black screen {(R,G,B)=(0,0,0)}, which has the lowest luminance, was displayed, and the luminance

at the center of the screen was measured five times.

50


• A white screen {(R,G,B)=(255,255,255)}, which has the highest luminance, was displayed, and

luminance at the center of the screen was measured five times.

• The positions in which luminance was measured were the front and a diagonal 45-degree position

relative to the display.

The specifications of the display are shown in TABLE 1. The measurement conditions are shown in TABLE 2. FIG.

2 shows the plans of the experiment and a photo.

TABLE.1:Specifications of display.

Display1

NEC VersaPro

VY11F/GL-R

Display2

Lenovo X41Tablet

Display3

Sony VAIO

VGN-SZ94PS

Display4

NEC Sheild PRO

FC-N22A

Display5

Sony XEL-1

Picture

CPU Intel Pentium M

1.1GHz

Intel Pentium M

1.6GHz

Intel Core2 Duo

T7800 2.6GHz

Intel ULV U7500

1.06GHz

RAM 512MB 1.49GB 2GB 2GB

Graphic Memory

(VRAM)

ATI MOBILITY

RADEON 7500

(32MB)

Intel 915GM Express

(96 MB)

NVIDIA GeForce

8400M GS

(256MB)

Intel GMAX3100

Weight (kg) 0.855 1.88 1.75 2.5 2.0

Display size (inch) 10.4 12.1 13.3 12.1 11

Time to market Aug., 2004 Jul., 2005 Dec., 2007 Aug., 2008 Oct., 2008

TABLE. 2: Measurement conditions.

Dark-room experiment Outdoor pilot experiment Outdoor experiment

Illuminance on the ground (lx) 0 22,030 – 107,800 29,470 – 83,300

Measurement time Jul. 4, 2009, at 20:00 – 22:00 Jul. 3 and 6, 2009, at 8:00 – 11:30 Jul. 9, 2009, at 9:45 – 12:00

Weather Fine weather Obscured sky Obscured sky

Latitude, Longitude 34.822623, 135.522781 (Osaka, Japan)

FIG. 2: Plan of dark-room experiment (upper left) and outdoor experiment (upper middle) (o: display power-off; k:

black image; w: white image); Photo of outdoor experiment (right).

51


3.2 Darkroom experiment

3.2.1 Experiment method

The method of the experiment outdoors was the same as described in Chapter 3.1. It was confirmed that the

illuminance of the darkroom was 0 lx by using the illuminance meter.

3.2.2 Result

The measurement was carried out on each pattern five times. The mean value of the value obtained for the five

measurements was used for the analysis. FIG.3 shows the luminance comparison on each display. FIG.4 shows the

contrast ratio comparison on each display. FIG.5 shows the viewing angle comparison on each display.

The value of the screen luminance when power was off was the lowest in each display, as shown in FIG.3. Each

value was close to 0. When the screen was black, Ld5 indicated the same value as the luminance when power was

off. This shows that jet-black can be expressed with the display5 made by the organic EL display. The luminance

value from the display1 to the display4 was higher than that of the display5 at the black screen. This is because it is

difficult to completely remove the influence of the backlight from the screen since it forms a basic part of the

principle of operation of the liquid crystal panel. When the screen is white, the Ld1 used as a display of the MR

system indicated 77.854cd/m2, , which was the lowest value. Ld4 indicated 394.48cd/m2 , which was the highest.

The contrast ratio CRd1-CRd4 of the liquid crystal display (Display1-4) was 346.9-577.4, as shown in FIG.4. On the

other hand, CRd5 of the organic EL display (Display5) was 147857.1, which was a considerably high value.

The viewing angle of the display was 2.21-5.33, as shown in FIG.5. The value of VAd5 was the lowest with 2.21.

FIG. 3: Dark-room luminance (Ld) comparison on each display.

FIG. 4: Dark-room contrast ratio (CRd) comparison on each display.

52


FIG. 5: Dark-room viewing angle (VAd) comparison on each display.

3.3 Outdoor pilot experiment

Outdoors, there are many parameters that make it difficult to obtain a constant result. Such experimental problems

were clarified by conducting a pilot study using display2 and display3.

3.3.1 Illuminance change outdoors

The illuminance of the ground was measured in pilot experiment 1. In regard to the experiment time, the illuminance

was measured for 2 hours and 30 minutes. This was done on July 3 and July 6, 2009. The time was from 9:00 to

11:30. The weather was overcast. The result is shown in FIG. 6.

FIG. 6 clearly shows that illuminance outdoors is not constant. On July 3, the illuminance meter indicated 35,100-

107,800lx. On July 6, the illuminance meter indicated 22,030-72,100lx. The illuminance changes outdoors influence

the luminance of the display. It is thought this change is caused by changes in the weather and the difference in the

time of the experiment. It is possible to select dates when the weather is steady, and to shorten the time of the

experiment.

FIG. 6: Illuminance change in outdoor.

3.3.2 Influence of sunlight

The influence of sunlight was examined in pilot experiment 2. To reduce the influence on the luminance of the

display, the display showed a black screen, and a black board was set up in the background. Luminance was

measured from the front, from a diagonal 45-degree position, and from a diagonal 135-degree position relative to the

display. The result is shown in FIG. 7.

53


FIG. 7 clearly shows that the luminance of the display measured from the diagonal positions was higher than that

measured from the front. This is thought to be due to specular reflection when measuring diagonally because of the

relation to the angle of incidence of sunlight. The luminance of the sun is about 2×109cd/m2, and it is overwhelming

compared with the brightness of the display. The display was set up with the sun behind it so that the influence of

direct sun could be reduced.

FIG. 7: Influence of sunlight (-fk: measurement from the front; -s1k: measurement from the diagonal 45 degrees; -

s2k: measurement from the diagonal 135 degrees).

3.3.3 Influence of ambient surroundings

The influence of the ambient surroundings was examined in pilot experiment 3. The display was set up with the sun

behind it, taking account of the result of pilot experiment 2. Then, the display was changed to a black screen, and

measured from the front. Different ambient surroundings that were reflected on the display were used and measured,

namely an unchanged state, a white board, and a black board. The result is shown in FIG. 8.

FIG. 8. clearly shows that the luminance of the display was the highest with the white board, and lowest with the

black board. That is, the ambient surroundings influence the luminance of the display. It is possible to stabilize the

scenery that is reflected on the display. The black board, which caused the least reflections, was set as the ambient

surroundings.

FIG. 8: Influence of ambient surrounding (-fn: a state without change; -fw: a state with a white board; -fk: a state

with a black board).

As a result of these considerations, the following experimental conditions were applied:

• The change in the illuminance was noted.

54


• The display was set up with the sun behind it.

• The black board was used for the background reflected on the display.

3.4 Outdoor experiment

3.4.1 Experiment method

The method of the outdoors experiment was the same as that described in Chapter 3.1. The illuminance of the

ground was measured at the same time as measuring the luminance.

3.4.2 Result

FIG.9 shows the luminance of each display and the illuminance of the ground. When the display was set to a black

screen, the luminance was 224.3-239cd/m2 in display1, 75.96-81.6cd/m2 in display2, 99.98-103.9cd/m2 in display3,

229.9-241.5cd/m2 in display4, and 243-258.3 cd/m2 in display5. All the values were higher than that of the

darkroom. It is thought that the outside light influenced this. When the display was set to a white screen, the

luminance was 320.2-337.3 cd/m2 in display1, 187.9-199.5 cd/m2 in display2, 229.4-231.5 cd/m2 in display3, 677.4-

726.5 cd/m2 in display4, and 326.4-333.6 cd/m2 in display5. All the values were higher than that of the darkroom. It

is thought that the outside light influenced this.

The ambient contrast ratio CRbn of all displays was 1.314-3.010 from FIG.10. The viewing angle VAbn of all displays

was 0.822-1.919 as shown in FIG.11.

55


FIG. 9: Outdoor luminance (Lb) and the illuminance comparison on each display (-fo: display power-off; -fk: black

screen; -fw: white screen).

FIG. 10: Outdoor contrast ratio (CRo) comparison on each display.

56


FIG. 11: Outdoor viewing angle (VAo) comparison on each display.

4. DISCUSSION

Outdoors, there are many factors that make it difficult to obtain a constant result. In this study, to decrease the

variation factor, the illuminance change outdoors, the influence of sunlight, and the influence of the ambient

surroundings were considered, as described in Chapter 3.3. Also, experimental methodology that reduced these

variation factors as much as possible was used in Chapter 3.4.

When FIG.9 is compared with FIG.3, the luminance L of the displays differs greatly in the dark room and outdoors

in the sunlight. The luminance of a white screen is higher than that of a black screen in each display though it is

necessary to remember that the illuminance can be uneven. Moreover, the luminance of a switched off screen is

higher than that of a black screen, excluding display5. The luminance of a white screen is higher than that of the

darkroom. This reason is that the influence of the outside light is received. Whether the ratio of the luminance of the

outdoor display is an influence of the luminescence of the liquid crystal display or of the outside light cannot be

judged from the measuring method of the luminance used in this study. When FIG.10 is compared with FIG.4, the

contrast ratio CR also differs greatly in the dark room and outdoors in the sunlight. One of the causes of this is that

the luminance of a black screen is high compared to the influence of the outside light. When FIG.11 is compared

with FIG.5, the viewing angle VA also differs greatly in the dark room and outdoors in the sunlight. As Chapter 3.1

described, when evaluating the display used outdoors, measurement of the ambient contrast ratio is important.

This experiment was executed in a certain specific time. That is, since outdoor illuminance constantly changes, it is

difficult to reproduce the result of the experiment. Therefore, it is necessary to build a display evaluation system

which can reproduce and measure outdoor daylight conditions.

5. CONCLUSION AND FUTURE WORK

The results achieved in the present study are as follows:

• The outdoor use of digital tools such as MR is anticipated, and there is a problem that the screen is

hard to see when a normal display is used. To clarify the reason for this, luminance, the contrast ratio,

and the viewing angle of the display were evaluated.

• Five displays were measured using a luminance meter, and as part of the luminance data, the contrast

ratio, and the viewing angle of each display were acquired in the darkroom.

• Outdoors, there are many factors that make it difficult to obtaining a constant result. To decrease the

variation factors, the illuminance change outdoors, the influence of sunlight, and the influence of the

ambient surroundings were considered. Also, using experimental methodology that reduced these

variation factors as much as possible, data on luminance, the contrast ratio, and the viewing angle of

each display were acquired outdoors.

Future works could investigate the following areas:

• Whether the ratio of the luminance of the outdoor display is an influence of the luminescence of the

liquid crystal display or of the outside light cannot be judged from the luminance measuring method

57


used in this study. A clear relation was not obtained although clarification was attempted in the

experiment by measuring luminance in the switched off state. It is necessary to examine the character

of the material that composes the display.

• To understand the outdoor characteristics of each display, the construction of an evaluation system

that artificially produces the outdoor environment is thought necessary.

6. ACKNOWLEDGEMENT

The author would like to thank Mr. Wei Cheng Lin and Mr. Tian Zhang who are research students of Osaka

University for supporting these experiments.

7. REFERENCES

Behringer, R. et al. (2000). "A Wearable Augmented Reality Test-bed for Navigation and Control, Built Solely with

Commercial-off-the-Shelf (COTS) Hardware.", Proc. 2nd Int’l Symp. Augmented Reality 2000 (ISAR 00),

IEEE CS Press, Los Alamitos, Galif., 12-19.

Baillot, Y., Brown, D., and Julier, S. (2001). "Authoring of Physical Models Using Mobile Computers.", Proc. Int’l

Symp. Wearable Computers, IEEE CS Press, Los Alamitos, Galif.

Feiner, S., MacIntyre, B., Höllerer, T., Webster, A. (1997). "A touring machine: Prototyping 3D mobile augmented

reality systems for exploring the urban environment.", Personal and Ubiquitous Computing, Volume 1,

Number 4, Springer London, 208-217.

Fukuda, T., Kawaguchi, M., Yeo, W.H., and A. Kaga, A. (2006). "Development of the Environmental Design Tool

"Tablet MR" on-site by Mobile Mixed Reality Technology.", Proceedings of The 24th eCAADe (Education

and Research in Computer Aided Architectural Design in Europe), 84-87.

Fukuda, T. (2009). "Analysis of a Mixed Reality Display for Outdoor and Multi-user Implementation ", 4th

ASCAAD Conference (Arab Society for Computer Aided Architectural Design), 323-334.

Julier, S. er al. (2000). "Information Filtering for Mobile Augmented Reality.", Proc. Int’l Symp. Augmented Reality

2000 (ISAR 00), IEEE CS Press, Los Alamitos, Calif., 3-11.

Kaga, A., Kawaguchi, M., Fukuda, T., Yeo, W.H. (2007). "Simulation of an Historic Building Using a Tablet MR

System.", Proceedings of the 12th International Conference on Computer Aided Architectural Design

Futures, Sydney (Australia), 45-58.

Kuo, C.G., Lin, H.C., Shen, Y.T., Jeng, T.S. (2004). "Mobile Augmented Reality for Spatial Information

Exploration.", Proceedings of The 9th International Conference on Computer Aided Architectural Design

Research in Asia (CAADRIA2004), 891-900.

Onohara, Y., Kishimoto, T., (2005). "VR System by the Combination of HMD and Gyro Sensor for Streetscape

Evaluation.", Proceedings of The 10th International Conference on Computer Aided Architectural Design

Research in Asia (CAADRIA2005), vol. 2, 123-128.

58


A PROPOSED APPROACH TO ANALYZING THE ADOPTION AND IMPLEMENTATION OF VIRTUAL REALITY TECHNOLOGIES FOR MODULAR CONSTRUCTION

Yasir Kadhim, Graduate Research Student

Department of Civil Engineering, University of New Brunswick, Canada;

[email protected]

Jeff Rankin, Associate Professor

and M. Patrick Gillin Chair in Construction Engineering and Management

Department of Civil Engineering, University of New Brunswick, Canada;

[email protected]

Joseph Neelamkavil, Senior Research Officer

National Research Council of Canada, Centre for Computer-assisted Construction Technologies

[email protected]

Irina Kondratova, Group Leader

National Research Council of Canada, Institute for Information Technology

[email protected]

ABSTRACT: To achieve successful adoption and implementation of process technologies in the construction

industry requires a better understanding of practices of innovation management. Defining innovation as the process

of applying something new, a research project is being undertaken to contribute to a better understanding of its

concomitant practices. The project focuses on virtual reality (VR) technologies within a specific application of

modular construction. From a potential adopter’s perspective, the process of technology adoption and

implementation is often less than satisfactory. The research project is addressing this by furthering the

understanding of the innovation process in a series of case studies. This paper presents work in progress to this end

by providing the background on assessing management practices, the link between VR technologies and modular

construction and early results of case study activities. To date, providing the functionality to enhance

communication has been identified as the best fit for the case studies of applying virtual reality technologies to the

process of modular construction engineering and management. The conceptual framework of assessing innovation

management practices that employs the concept of capability maturity is presented as a predictive indicator for the

adoption and implementation that is to follow.

KEYWORDS: Virtual reality, modular construction, innovation management, technology adoption

1. INTRODUCTION

Many practitioners and researchers alike agree that the architectural engineering and construction (AEC) industry

can improve its overall performance (measured in terms of cost, time, safety, quality, sustainability, etc.) by creating

a better business environment that encourages innovation. Innovation is defined in this context as “application of

technology that is new to an organization and that significantly improves the design and construction of a living

space by decreasing installed cost, increasing installed performance, and/or improving the business process (e.g.,

reduces lead time or increases flexibility)“ (Tooles 1998).

The research described focuses on process technologies within the AEC industry as a class of innovations. Process

technologies are loosely defined as any tool or technique that supports the management of a civil engineering project

during execution from concept, through design, construction and operation, to decommissioning. This focus area

presents some interesting challenges and some corresponding gaps in the knowledge area. From a potential

adopter’s perspective, it is difficult to objectively assess process technologies for adoption and implementation as

there are not many decision-making tools and techniques for industry to properly identify needs and match

corresponding solutions. Overcoming this challenge requires a direct link to performance, whether at the

organization, project or industry level, whereas currently, the focus has been on operational savings. For example, it

59


is easy to measure the time savings realized by recording information electronically versus on paper; however, an

assessment of the knock-on positive effects on performance, by having this information conveniently archived, is a

bit more difficult to measure. Some of the questions to answer include: how do we improve the performance of the

AEC industry through the effective development and appropriate adoption and implementation of process

technologies; what are the techniques to support practitioners in the analysis of new process technologies; and what

contributes to a strategy for increasing the impact and rate of process technology adoption within the AEC industry?

The approach that has been taken is to assess the performance of the process of construction while taking into

account the management practices being applied. A modest research project is being conducted jointly by the

University of New Brunswick’s Construction Engineering and Management Group (UNB CEM) and the National

Research Council of Canada’s Centre for Computer-assisted Construction Technologies (NRC-CCCT). The short

term research objectives are to study the implementation of a specific advanced process technology (i.e., virtual

reality technologies) for a specific scenario in the industry (i.e., modular construction). The research is also intended

to contribute to a broader research program of more formally assessing the impact of innovation management

practices on industry performance. The research project hypothesis states that the maturity of management practices

at various levels within an organization, with respect to process technologies, can be measured and correlated with

the performance in the adoption of technologies.

The research project contains the following objectives: 1) defining the basic competency requirements beyond the

standard construction engineering and management domain, for both modular construction and virtual reality

technologies;, 2) determining the basic challenges in the adoption and implementation of modular construction and

virtual reality technologies;, 3) establishing the details of a case study through the definition of usage scenarios for

the application of virtual reality technologies to modular construction;, 4) developing the technological environment

required for the usage scenarios through the configuration of existing technologies, and 5) capturing the case study

and assessing the use of virtual reality technologies to further a definition of innovation management.

The paper reports on work in progress by first providing background on the approach to assessing performance and

practices, topics of virtual reality (VR) technologies and their advantages and modular construction approaches and

their advantages and challenges. The details of the industry case studies being used in the research project are

provided, followed by a description of the methodology being used to assess the adoption and implementation of the

technology contributing to the assessment of innovation management practices at an organizational level.

2. ASSESSING PERFORMANCE AND PRACTICES

To place this research in the context of assessing performance in the construction industry, Figure 1 depicts a high

level process view of construction (Fayek et al. 2008). Measuring the performance of the process at some level of

granularity (e.g., activity, project, organizational, sector, industry) typically measures the ratio of outputs to inputs

(A to A) and the extent to which objectives are achieved (C), under a given set of conditions (B), while employing a

set of practices (D). The research described in this paper explores innovation management practices (D) and it does

so at the organizational level of granularity. The aggregation (e.g., to a sector level) and/or specialization (e.g., to an

activity level) of the assessment is not covered in the scope of the framework developed. In order to study this, a

specific innovation (VR technologies) and scenario for application (modular construction) is required.

Process maturity modelling gained its greatest attention in the software manufacturing industry (Finnemore et al.

2000) and is based on the earlier concepts of process improvement such as the Shewhart plan–do–check–act cycle,

as well as on Philip Crosby’s quality management maturity grid which “describes five evolutionary stages in

adopting quality practices” (Crosby 1979). Researchers at Carnegie Mellon University used this concept in the

development of the Capability Maturity Model (CMM) (Paulk et al. 1995). CMM highlights the five thresholds of

maturity through which a process must transition in order to be sustainably improved. Initially, a process is (1)

chaotic or ad-hoc and must be made (2) repeatable, after which it must be (3) defined or standardized. The process

must then be (4) managed, i.e., measured and controlled. Ultimately, the process must be (5) optimized, i.e., it must

be continuously improved via feedback and through the use of innovative ideas and technologies. The assessment of

the maturity of a process at the organizational level entails determining the extent to which the process is defined,

managed, measured and controlled; and this is commonly achieved by observing the practices within the

organization. A more general definition is that maturity may be viewed as a combination of actions, attitudes, and

knowledge rather than constraining the definition to a single set of actions or procedural norms (Andersen and

60


Jessen 2003). Closer to the construction industry and management of projects are more recent maturity models that

include the Project Management Process Maturity (PM)2 Model (Kwak and Ibbs 2002), the Standardised Process

Improvement for Construction Enterprises (SPICE) Model (Sarshar et al. 1998), and the related research area of

learning organizations in construction (Chinowsky et al. 2007).

FIG. 1: A conceptual model for assessment of the industry (from Fayek et al. 2008)

The assessment of the maturity of innovation management practices builds upon previous work on this topic. Willis

and Rankin (2009) have defined a maturity model to assess management practices within the construction industry

at an industry level. The model uses a three level construct for maturity where a practice is: (1) immature, in that it is

ad hoc in its application, (2) transitional mature, in that it is defined and repeatable, and (3) mature, in that it is

measured and improved. The levels correspond to a range from zero to one; zero, where it does not exist, and one,

where it is mature.

2.1 Virtual Reality in Construction

The application of virtual reality technologies and tools in construction has been one of the widely discussed and

researched topics in the construction industry scholars’ community during the past decade. Many perceive the tools

offered by VR to be very useful in assisting with visualization and in enhancing the understanding of spatial as well

as temporal aspects of the construction process. Those specific advantages of the application of VR tools bring about

overall benefits in the general planning and scheduling of construction projects.

The previous research, completed in the assessment of virtual reality for construction application by many

researchers, has proved that the use of 3D and VR walk-thru technologies could assist in the development of more

complete and accurate schedules through having a significant impact on the schedule review process (Songer et al.

2001). Another area of application includes the use of 4D tools for educating or training purposes. Messner and

Horman (2003) proved through a study that 4D assisted in a better understanding of construction plans, especially

for inexperienced personnel.

Whisker et al. (2003) performed experiments to study the application of 4D in an Immersive Virtual Environment

(IVE). It was shown in the experiments that the use of IVE assisted in reducing the planned schedule duration by

28%, identifying constructability issues and evaluating schedule dependencies. A rather practical study application

was developed for a strategic decision support system for virtual construction, (VIRCON) (Dawood et al. 2005). The

main target of the system was to enhance the ability to trade-off temporal aspects with spatial aspects in order to

come up with a more developed construction schedule. More recently, Dawood and Sikka (2008) studied the

benefits of 3D/4D as a communication tool. Their study and practical experimentation proved that the use of 4D

models is a more efficient tool for communicating and interpreting construction plans compared to traditional 2D

CAD. The study also proved that the findings are valid even amongst experienced personnel who are accustomed to

using 2D CAD.

61


Thus, the authors have identified four major categories of benefits of VR technology application for the construction

industry. These serve as initial categorizing guidelines for the assessment of the specific VR technology adoption

within a modular construction context. The four areas of possible improvement are: scheduling, space planning,

communication, and training and educational applications.

2.1.1 Scheduling

The development of construction schedules is one of the most comprehensive tasks in construction planning as it

requires the input of a variety of information, as well as involvement of many participants, especially for larger

projects. Due to the importance and complexity of construction scheduling, various efforts have been put into

providing aiding tools and applications and the integration of some of those tools for more efficient scheduling.

Various software applications have been developed to assist in generating more complete and accurate schedules

and to establish formal scheduling approaches. The integration of such software with 3D or 4D visualization has

been studied and is expected to further boost the scheduling capabilities of construction managers. The use of 3D

and VR walk-thru can assist in the creation of more “correct” schedules (Songer et al. 2001). Correctness includes

three characteristics: completeness (measured by the number of missing activities), valid activity relationships

(measured by errors such as physical impossibilities, redundant or looped relationships) and reasonable activity

duration. Less significantly, but still advantageous, 3D and walk-thru also help to create “good” schedules (Songer

et al. 2001). Goodness metrics include end date, critical path, and float and resource fluctuation. The use of 4D can

also assist in detecting scheduling logical errors more frequently, faster and with fewer mistakes and can also

compensate for a lack of practical experience (Kang, Anderson and Clayton, 2007).

Studies proved that VR technologies are a valid and comprehensive tool that provides not only a better

understanding of spatial and temporal aspects (hence avoiding errors), but also identifies more areas of

improvement, and boosting confidence in generated schedules.

2.1.2 Space planning

Space is considered one of the more critical resources in construction projects. Due to the fact that many participants

or crafts are involved at any specific time in an average to large size project, the ability to plan for space distribution

is difficult and often spread amongst participants. If effective communication is lacking, this planning spread would

face confusion, unexpected space clashes, delays and increased costs.

Naturally, VR technologies will provide a better understanding of space through visualization. Project participants

can effectively analyze problems regarding sequential and spatial conflicts prior to actual construction operations.

Research has covered this area of VR applications. VIRCON (Dawood et al. 2005) has a wide set of space planning

tools (some produce plans automatically through simulation and databases and others used as aiding tools for

manual planning). These tools include assigning plants and temporary works to space plans, checking for possible

space clashes, marking-up available space and distributing tasks over the life time of the space. The use of 3D

modelling has long been used for plant design to check for space requirements and it is only natural for 3D/4D and

VR technologies to be developed for the analysis and planning of space in actual construction site settings. Better

space planning can lead to valuable benefits in other ways such as in the reduction of overall project duration and in

maintaining good relationships amongst project participants.

2.1.3 Communication

Improving the efficiency of communicating construction plans and schedules could be argued as the essential

advantage and from which other benefits of using VR stem. Visualization, in concept, is the process of displaying

information in order to assist in understanding and evaluating it. Improved communication could lead to time

savings through reduction in reconstruction. This is due to the increased ability to illustrate logic, sequence and

interrelationships among construction tasks and products. Also better communication leads to increased confidence

in all aspects of planning.

The traditional use of 2D designs and plans introduces unnecessary secondary stages and tasks within the planning

process. It can also cause confusion when information is transferred among participants. In addition, the use of

traditional 2D plans requires a variety of information storage media and hence makes the planning process

vulnerable to misinterpretations and loss of information. Although the creation of 3D and 4D models and plans

62


demands more resources initially, if applied appropriately, these tools could save on the total time and effort for

overall planning (Kang, Anderson and Clayton, 2007).

2.1.4 Training and educational application

The use of VR technologies as a visual communication enhancer can greatly improve the ability to learn and gain

rapid experience related to various construction management and design skills. Advanced visualization tools can

also assist students or trainees by providing them with the chance to assess their decisions and their impact. This

option is not likely to be feasible in an actual site setting as training is mostly observatory due to the high cost of

errors. Even experienced personnel might not be as willing to try a new approach or method due to the conservative

nature of the construction industry. VR, along with simulation, has proved to provide an effective remedy such that

learning is more proactive and less traditional.

Several experiments were conducted by Messner and Horman (2003) to measure the added value for students when

using 4D CAD tools when reviewing construction plans. Several conclusions were drawn: the benefits of using 4D

CAD includes improving the understanding of sequencing issues and their importance, improving the ability to

evaluate important flow concepts, quickly understating complex building models and gaining experience at faster

rates.

2.2 Modular Construction

Recent increasing concentration on aspects such as cost, schedule and labour issues within the construction industry

has made prefabrication, preassembly and modularization in construction more feasible than ever before. In addition

to those drivers, advances in information technologies and construction engineering software would appear to make

pre-work (a term that encompasses the aforementioned three similar construction methodologies) easier to apply and

with reduced accompanying risks.

Modular construction is, by definition, a term that stands for the systematic approach of breaking down a

construction product design into complete systems that are fabricated off-site with the involvement of multiple

trades in a controlled environment, and then transported to the construction site and assembled with minimal effort

(in comparison with their fabrication) (Haas and Fagerlund, 2002). The concept of modular construction is flexible;

the implementation of modular construction strategies and methodologies cover a wide spectrum of applications and

for a variety of applications (Gibb 1999).

There are various drivers and benefits to the application of modular construction. Some of those benefits are easy to

recognize and some vary, depending on the application scenario. The major drivers to the use of modular

construction relate to the general parameters of any construction project which are cost, schedule, quality and safety

(Gibb and Isack 2003). Other secondary, but still significant parameters include the environment, maintenance,

design, secrecy and others. In terms of cost, there are many possible ways through which the application of modular

construction can result in savings. Increased productivity of workers, due to the controlled and more organized

environment, as well as easy access to tools and equipment in modular construction, by nature, reduces project costs.

Another major source of decreased overall costs is the reduced cost of onsite labour. Schedule drivers are also one of

the most significant when making the decision to modularize. A modular context is by nature more repetitive and

includes fewer variables to account for when planning and scheduling projects. This repetitive character as well as

the controlled environment also translates to more efficient and less costly quality control and enhanced safety (Gibb

and Isack 2003).

Modular construction appears to have gained greater consideration during times of high construction demand and

activity industry wide. For example, there was a significant rise in modular residential construction activity during

the economic boom that followed the energy crisis of the 1970’s. Looking at the current economic conditions and

the construction industry overall, one can deduce that there will be an opportunity to increase the demand for

modular construction to provide for efficiency gains and the ability to meet infrastructure demands worldwide. Even

as the current conditions stand, modular construction can still be recognized as being on the rise as international

efforts have incorporated it into their industry initiatives and the current NRC Construction Key Sector Group has

identified prefabrication, preassembly and modularization as a new effort in its strategic plan (NRC 2006).

Observing the variety of benefits that could be achieved through modular construction, it could be presumed that its

application should be highly rewarding and attractive. However, there are various implications and issues that hinder

63


the spread of modular construction within the construction industry worldwide. Modular construction represents a

significantly different approach when compared to traditional onsite construction. This difference raises new issues

and impediments that influence the decisions to undertake modular construction. One area of major concern is

related to engineering requirements. Depending on the extent of pre-work, it may be necessary to complete 90% of

the engineering design prior to construction, as opposed to the 40% generally necessary for conventionally built

projects (Tatum et al. 1987). In addition to the need for early design completion, there are additional factors to be

addressed. The specific dimension or loading limitations, due to transportation constraints, are one of the most

prominent factors in modular construction (Fussell 2007).

Scope flexibility is also expected to be decreased with modular construction. A well defined scope is essential for

effective project planning and to avoid any changes later in the life cycle of the project; note that changes are

significantly more costly in a modular construction scenario when compared to traditional methods. Finally, the

increased demand for effective coordination and communication among participants is a barrier to modular

construction. The distribution of the work load, formation of work breakdown structures, progress monitoring,

scheduling and organizational structures might all need alterations from the traditional sets to provide for a

successful modular construction project. All those alterations require highly effective communication and

collaboration among participants (Prieto 2008).

2.3 Opportunity of VR Application in Modular Construction

Observing the aforementioned impediments of modular construction, it can be concluded that VR application

opportunities exist and can assist in facilitating modular construction. Haas and Fagerlund (2002) and others

recognized the importance of computer integration and technological advances such as 3D CAD and the possibility

of its application in modular construction. VR technologies go beyond the capabilities of 3D design to include

aspects such as time and more effective communication. Also, the application of VR would be facilitated due to the

need to have a significant proportion of the engineering design completed prior to commencing construction

activities. The completed engineering design would provide for all the input to generate useful virtual reality tools

that can be applied to enhance performance.

Enhancing physical interface management could be one of the most direct and initial advantages of using virtual

reality technologies. The visualization enhancement provided by VR technologies would assist the engineers in

assessing the complex modules and plan for efficient assemblies in terms of fabrication as well as installation.

Another aspect that can be improved is spatial planning for transportation concerns. While trading off with temporal

aspects, engineers can implement virtual reality technologies to generate efficient transportation schedules and

plans.

Mitigating the reduced scope flexibility could also be another advantage of VR application. VR allows for effective

communication of plans with other project participants during the early stages. This would allow owners to have a

better idea of the end product and perhaps suggest alterations prior to the initiation of work. This would also

increase confidence and improve the relationship with the owner. Increasing the efficiency of communication will

also benefit the overall project at which coordination of multiple sites is needed. The distribution of information

among participants can be enhanced and also VR technologies can provide the interface for drawings and

visualizations needed in modular construction projects, in addition to traditional plans.

3. INITIAL CASE STUDY FINDINGS

The VR technologies to be implemented are developed in partnership with the NRC-CCCT. In addition, two

industry participants were identified and secured for the practical application of the VR technologies and adoption

assessment. The two companies offer a unique perspective on varying prefabrication technologies and applications

which should assist in having an unbiased and more comprehensive understanding of the prefabrication industry and

its technology challenges. Following meetings with the industry participants to initially identify the direction and

general needs and challenges within the prefabrication construction industry, further analysis was performed to

identify which VR technologies are available and match the needs and resources of the industry. The limitations of

VR technologies were identified, as well as the challenges to their practical application.

64


3.1 Identified Challenges

Industry practitioners in modular construction were identified through a session organized by the NRC Institute for

Research in Construction to explore the creation of a national network for modular construction (NRC 2009). The

two companies were selected primarily due to the diversity of their operations, from several perspectives. The first

company specializes in mass production of wooden wall panels. The company utilizes an imported system which is

mostly automated for the fabrication of specifically designed wall and floor panel sections. The company fabricates

wall panels for large housing projects and was initially a traditional general contractor which decided to make the

transition to prefabrication about a year ago. The second company specializes in the production of composite wall

panels. The panels are fabricated using a patented design utilizing both concrete and cold rolled steel. The company

aims more at securing larger projects which provide for more feasible investment. Originally, the company

specialized in manufacturing machinery used for fabrication of steel office supplies. Following a decline in the

manufacturing industry, the company decided to go into the construction business and apply some of the

manufacturing principles to gain an advantage. The company still attempts to manufacture their own machinery for

specific tasks such as welding, whenever possible, although that is limited to resources and investment feasibility

(size of projects). Table 1 provides an overview of each company, illustrating the unique and different perspective

each offers.

TABLE 1: An overview of the case study participants.

Factor Company A Company B

Location Southern Ontario Southern Ontario

Company Background Construction (general contractor) background Manufacturing background

Product Wooden Panels Composite Panels (Concrete/Steel)

Main Market Housing Industrial

Source of Fabrication System Purchased In house developed / Patented

Scripted interviews were prepared with open-ended questions to collect data on the opportunities for VR previously

identified and the general structure and operations of each organization. The visits consisted of the interview with

senior management and a tour of the facilities. The results were then summarized and validated with the interview

participants. Each company offers a unique view of prefabrication in construction, where not only their processes

but also their approach towards prefabrication is rather different. Nevertheless, both identified the same two major

issues as their main challenges: communication and integration.

3.1.1 Communication

General communication and education issues with multiple players within the industry have been raised as a general

concern. Modular construction is relatively new in Canada as a significantly different and more progressive

methodology when compared to Europe, for example. This causes clients, architects and subcontractors to handle

modular construction, even in its most simple forms, with scepticism and worries of increased complexity in

assembly and reduced flexibility such as aesthetic options. Enhanced, more effective and practical communication is

needed to establish confidence within the industry and to educate the industry on the use and advantages of modular

construction. Some of the communication challenges that were identified were: ability to communicate the available

options to the architects and clients and the ability to communicate the assembly processes or functionality of the

end product to the architect, client and more importantly subcontractors who will be undertaking assembly on the

site. The communication issues identified were more directly related to opportunities for VR application tools to

enhance the efficiency of communication, and in some ways, promote the methods of prefabrication through

educating the participants.

3.1.2 Integration

Although not initially included in the scripted interview, the issue of integration was identified as one of the

significant issues causing delays within the production processes. Also, integration issues and complications were

found to exist on two major fronts: integration between architectural drawings and construction designers/detailers,

and the integration between completed design and the machinery used for the manufacturing of designed assemblies.

Existing integration issues are mainly an underlying form of interoperability. Interoperability is a varying and

widely discussed topic with continuing long term efforts to resolve case-specific interoperability issues. The general

approach to resolving interoperability issues, which was the same followed by the selected industry participants, is

65


to use an off-the-shelf application or software or one that is developed by a third party specializing in software. The

companies then attempt to standardize the software within their systems in order to limit any integration issues.

However, interoperability or integration issues and problems have been found to always exist and are rarely totally

remedied. Integration still is one of the major bottlenecks within production processes for two major reasons: the

involvement of various participants, making it difficult to have one standardized approach adopted by all involved.

Second, because of the fact that the prefabrication industry’s demand for integration solutions is limited in volume,

few software developers are willing to offer efficient and tailored solutions for their integration problems, and even

those offered are charged in total to the single specific order, which makes them rather costly.

3.2 Virtual Reality Technologies Solution

Following the identification of the main challenges faced by the industry participants, the focus was then on

assessing the VR technologies available, their limitations, and how they could be tailored to fit the needs of the

industry. An overall assessment of the NRC-CCCT facilities in London, Ontario was conducted in order to evaluate

its VR technology capabilities. The VR facilities include state of the art environments and hardware such as

separately functioning theatre, cave, and motion capture environments, as well as other more portable options such

as the 3D scanner, LCD’s and compact and powerful processors. In addition, the VR tools and technologies

available have been implemented in various manufacturing as well as construction-oriented applications. Alongside

sophisticated modelling and visualization for the auto manufacturing industry, examples of previous projects include

training environments for crane operators and motion capture of human/environment interactivity. The following is

a summary of the significant aspects of the available VR technologies related to the need identified:

• The use of portable VR hardware has very few and case-specific limitations which are not likely to exist

within the scope of the usage scenarios to be applied.

• In terms of software, various programs are available for use in multiple applications such as basic 3D

design software, more advanced graphics software, animation and interactivity software, as well as in-

house programmed software for specific applications.

• Software programs can be integrated and combined for a more complete and practical usage scenario.

• Previous work on creating a VR model for a basic demonstrative construction application has been done

and the findings of the project (in terms of resources required and effort) are considered.

• It was identified that in order to have a real VR application, interactivity was a vital element of the VR

tools to be offered to the industry participants.

• In order to have a feasible application, it is necessary to have tools that can be used on a continuous basis.

For example, directories of components that can be repeatedly used instead of single, custom or project-

specific applications.

Therefore, taking into consideration the needs identified and the capabilities of the technology, the main focus of the

project is on the application of VR technologies as communication process technologies and tools with all types of

participants. Minor assessment and consideration of integration issues will still be considered, however, integration

issues would be addressed strategically rather than technically. The application of the VR technologies will include

interactive elements and not only advanced illustration tools. The creation of reusable tools is an essential aspect and

that can be accomplished through creating directories of objects or tools which facilitate continuing use of the VR

technologies rather than keeping it case-specific.

4. NEXT STEPS

The immediate next steps in the research project consist of developing the VR technology prototype and assessing

the organizations’ capacity for implementation and adoption through an assessment of innovation practices. The

development of the appropriate VR technologies is relatively straightforward with the assistance of NRC-CCCT

expertise. The assessment of an organization’s capacity for innovation requires the completion of an appropriate

framework.

66


4.1 Developing a Technology Solution

In order to establish an adequate technological environment suited for the practical application of the VR

technologies by the industry participants, four major steps need to be completed: 1) gather additional information

and feedback regarding the specific needs of the participants and their use of the VR tools, 2) acquire all detailed

design information from the participants needed to create the VR environment, 3) start with the creation of the basic

3D models and components of the VR environment, and 4) complete the VR tools by introducing interactivity

elements to the 3D environment, as well as adding the element of time for the creation of 4D models.

There are various options available for the creation of the 3D components in terms of software. On the other hand,

the use of hardware such as the 3D scanner is not likely due to the difficulty of establishing smart models using

these tools (models that include groupings, components, hierarchies … etc.). The NRC-CCCT facilities and

technical support enables importing 3D models made using any of a number of software options such as 3D CAD,

Maya, 3D Max and Google SketchUp. Due to the experience of the researcher with Google SketchUp, the relative

simplicity of the construction models, when compared to other manufacturing modelling, and the free availability of

the software (for both model creator and end user), Google SketchUp was selected to create the 3D environment.

However, the models created with Google SketchUp will be enhanced graphically using Maya at a later date before

incorporating interactivity and time elements.

4.2 Innovation Maturity Framework

Figure 2 depicts a conceptual framework of how innovation management practices will be assessed for the case

study companies. Innovation management has been broken down into factors that influence its outcome. The factors

will be grouped and each grouping is to be assessed based on the plan-do-check-act management cycle. The

assessment will be completed through a series of questionnaires structured to determine the level of maturity within

an organization. The maturity is reported with respect to the level achieved and remaining for improvement for each

management cycle step (e.g., bar chart) and can also be compared against other organizations in a benchmarking

exercise (e.g., radar chart).

FIG. 2: Conceptual framework for assessing the maturity of innovation management processes.

67


The case studies are being used as a step in validating the factors and groupings. A weighting of factors will then be

completed based on pair-wise comparisons by employing the analytic hierarchy process, where each step within a

grouping is weighted and then each grouping of factors is weighted. When completed for a group of experts, the

geometric mean of the results will be used to determine the contribution to the maturity scores. This allows for

analyses as presented in Figure 2. The chart is indicating the relative importance associated with each step within

innovation management along with a maturity score (achieved) and opportunity for improvement (remaining) at an

organizational level. This will then give a comparison with the level of success in implementing VR (performance

impact), however, a case study of two will not be significant, therefore a broader study will be conducted.

The paper is reporting on work in progress that is intended to support the development of a means of assessing the

innovation management practices of construction industry organizations. The results will be used to identify

correlations with performance and as a predictive tool for implementation and adoption of process technologies.

5. ACKNOWLEDGEMENTS

The authors wish to express their thanks to the case study participants for their time dedicated to this research

project. Appreciation is also extended to the National Research Council of Canada’s Centre for Computer-assisted

Construction Technologies and Institute for Information Technology who are supporting the research. Sponsorship

is also provided by the Natural Science and Engineering Research Council of Canada.

6. REFERENCES

Andersen, E.S. and Jessen, S.A. (2003). Project maturity in organisations. International Journal of Project

Management, Vol. 21, 457-461.

Chinowsky, P., Molenaar, K. and Realph, A. (2007). Learning organizations in construction. Journal of

Management in Engineering, Vol. 23, No. 1, (January), 27-34.

Crosby, P. B. (1979). Quality is Free: The Art of Making Quality Certain. McGraw-Hill Book Company. New York

Dawood N., Sikka S. (2008). Measuring the effectiveness of 4D planning as a valuable communication Tool,

Proceedings of the 7th International Conference on Construction Applications of Virtual Reality,

Pennsylvania State University, USA.

Dawood N., Scott D., Sriprasert E., Mallasi Z. (2005). The virtual construction site (VIRCON) tools: an industrial

evaluation. ITcon, Vol. 10, 43-54.

Fayek, A., Hass, C., Rankin, J., and Russell, A. (2008). Benchmarking program overview: CSC performance and

productivity benchmarking program, phase 1, CSC-CBR Technical Report P1-T12, Ottawa, Ontario, Canada,

23 pp.

Finnemore, M., Sarshar, M. & Haigh, R. (2000). Case studies in construction process improvement. Proceedings of

the Arcom Construction Process Workshop, Loughborough University, Loughborough.

Fussell, T. et al. (2007). Off-site manufacture in Australia: current state and future directions. A report for the

Cooperative Research Centre for Construction Innovation, Queensland University of Technology, 51 pp.

Gibb, A. G. F. (1999) Off-site fabrication: prefabrication, pre-assembly and modularisation. John Wiley and Sons,

ISBN 0470378360.

Gibb, A. and Isack, F. (2003). Re-engineering through pre-assembly: client expectations and drivers. Building

Research and Information, Vol. 31, No. 2, 146-160.

Haas C. T., Fagerlund W. R. (2002). Preliminary research on prefabrication, pre-Assembly, modularization and off-

site fabrication in construction. A Report to the Construction Industry Institute, The University of Texas at

Austin, Austin, Texas, USA, 76 pp.

Haas C. T., O’Connor J. T., Tucker R. L., Eickmann J. A., Fagerlund W. R. (2000). Prefabrication and preassembly

trends and effects on the construction workforce. A Report of Center for Construction Industry Studies,

Report No. 14, The University of Texas at Austin, Austin, Texas, USA.

68


Kang J., Anderson S. and Clayton M. (2007). Empirical study on the merit of web-based 4D visualization in

collaborative construction planning and scheduling, Journal of Construction Engineering and Management,

ASCE, June 2007, 447-461.

Kwak, Y.H. and Ibbs, C.W. (2002). Project management process maturity (PM)2 model, Journal of Management

in Engineering, Vol. 18, No. 3, (July), 150-155.

Messner J. and Horman M. (2003). Using advanced visualization tools to improve construction education,

Proceedings of the3rd International Conference on Construction Applications of Virtual Reality, Virginia

Tech, USA.

NRC (2009). Industrialization in building construction: report on the industry stakeholder meeting, Ottawa, April

24-25, compiled by Suthey Holler, National Research Council.

NRC (2006). National research council strategy 2006-2011, 28 pp.

Paulk, M., Weber, C., Curtis, B. & Mary-Beth, C. (1995). The capability maturity model: guidelines for improving

the software process. Addison-Wesley Longman Inc. USA.

Prieto, R. (2008). A great LEAP: modularization as a strategy, The Voice, The Official Magazine of the

Construction Users’ Roundtable, Fall, 47-52.

Sarshar, M., Hutchinson, A., Aouad, G., Barrett, P., Minnikin, J. and Shelley, C. (1998). Standardised process

improvement for construction enterprises (SPICE), Proceedings of 2nd European Conference on Product and

Process Modelling, Watford, UK.

Songer D.A., Diekmann J., Rasheed K. and Hays B. (2001). Construction scheduling using 3-D CAD and walkthru,

Construction Innovation, Vol. 1, 191-207.

Songer D.A., Diekmann. and Karet D. (2001). Animation-based construction schedule review, Construction

Innovation, Vol. 1, 181-190.

Tatum, C. B., Vanegas, J. A., and Williams, J. M. (1987). Constructability improvement using prefabrication,

preassembly, and modularization, Construction Industry Institute, The University of Texas at Austin.

Tooles, T.M. (1998). Uncertainty and home builders’ adoption of technological innovation, Journal of Construction

Engineering and Management, ASCE, Vol. 124, No. 4, 323-332.

Whisker V., Baratta A., Yerrapathruni S., Messner J., Shaw T., Warren M, and Rotthoff, E (2003). Using immersive

virtual environments to develop and visualize construction schedules for advanced nuclear power plants.

Proceedings of ICAPP '03, Cordoba, Spain, Paper 3271.

Willis, C. and Rankin, J. (2009). Maturity modeling in construction: introducing the construction industry macro

maturity model (CIM3), CSCE 2nd International Construction Specialty Conference, St. John’s

Newfoundland, May, 10 pp.

69


COLLABORATIVE 4D REVIEW THROUGH THE USE OF INTERACTIVE WORKSPACES

Robert M. Leicht, Ph.D., BIM Manager,

DPR Construction;

[email protected]

John I. Messner, Ph.D., Associate Professor of Architectural Engineering,

The Pennsylvania State University;

[email protected] and http://www.engr.psu.edu/ae/messner/

ABSTRACT: This paper presents quasi-experiments conducted during the validation process for a framework to

plan interactive workspaces titled INVOLVE. The validation is focused on the value demonstrated for different

interactive workspace components, focusing on interaction with the model, while teams perform a 4D schedule

review task. Interactive workspaces are spaces with potential for ubiquitous electronic interaction to allow for

enhanced communication and capture of information. In the experiment reported, 16 groups of students, with

typically 3 group members, were asked to review the 4D schedule of the Long and Foster Headquarters project in

Chantilly, VA. The 4D schedule reviewed during the task was a modified version of the contractor’s actual project

schedule to incorporate a total of 11 known schedule challenges. The focus of the study was to identify changes in

the students’ interactions when using an interactive workspace, and the differences that occur when model

interaction was limited through the interface or through the sharing of the interactive device and display. This

paper will provide results on the students’ perceptions of the value of the workspace and interaction through a post-

test questionnaire.

KEYWORDS: Interactive Workspaces, 4D Modeling, Interaction.

1. INTRODUCTION

Collaboration plays a central role in the design and construction of buildings, infrastructure and other facilities.

Recent developments in technology, such as building information modeling, virtual reality, and computer mediated

communication, have been shown to add value to this collaborative process. Previous work into virtual

environments (VE) has found that benefits from design and construction are focused on team design support and

decision making tasks (Gopinath, 2004). Despite this value, it is challenging to document the traits which are

essential in virtual environments and how these contribute to improved team collaboration. Defining these benefits

as they relate to specific characteristics provides a means for comparison between different VE setups, interactions,

and applications. To these ends we have undertaken a study to compare between two different variables within an

interactive workspace to study how teams value the use of the different tools which are available.

The study presented in this paper represents a step in the validation process for the INVOLVE Framework. The

INVOLVE Framework is intended to allow for the planning of virtual prototype use for the alignment of the

physical media with the needs of the virtual content and task for improved collaboration (Leicht, 2009). INVOLVE

has seven aspects which are interrelated for the collaborative and communicative aspects of using virtual prototypes:

I – Interaction – the physical mode of interaction between the user and the system

N – Network – the computing infrastructure to connect the system with internal and external systems

V – Virtual Prototype – digital representation of real world items for interaction, simulation, and testing

O – Organization – the team, the project structure, and the competencies needed to complete the task

L – Layout – the physical space and elements needed for teams to work together for a task

V – Visual Display – the location, resolution, and type of display system utilized to share information

E – Existential Collaboration – Authentic collaboration through shared goals, knowledge, and resources

71


The two traits of the interactive workspace to be studied are the visual display ability of more than one user to be

able to work in parallel to add and display information, and the value of having and virtual prototype interaction in

the form of the 3D model for a 4D schedule review.

2. USE OF INTERACTIVE WORKSPACES

The available literature offers evidence of improved performance when teams meet and work using improved

visualization tools and software within interactive workspaces. Fruchter (2005) studied a group of industry

members given a team’s project at a set point to discuss and generate ideas for future directions and challenges while

the students observe and ask questions. In the findings, Fruchter identified the unique aspect that the virtual

attendees are actually more engaged in the content due to their proximity to the developed material than the physical

attendees. Wang et al (2007) performed a comparative observational study to discern the role that software played

when using a virtual environment for team problem solving for a construction scheduling task, finding greater

enjoyment and interest from the student when meeting in an interactive workspace. Issa et al (2007) found added

value for student design projects when using the Interactive Collaboration Lab at the University of New Brunswick.

Students demonstrated that the lab enabled them to collaborate more effectively, make more educated decisions,

more effectively use their time, and produce higher quality work. These studies demonstrate added engagement and

greater value derived from meetings using interactive workspaces, but the specific impacts of hardware and interface

considerations are not the focus.

Some of these efforts have proven beneficial in field construction settings as well. Liston compared traditional

construction project meetings with project meetings using 4D visualization and identified increased efficiencies in

communications among team members (Liston et al, 2001). Khanzode found improved coordination among

specialty contractors on the Camino Hospital Project when the team developing the shop drawings for the duct,

piping, plumbing, and sprinkler systems were housed in the “Big Room” there was a shorter time and more

economic process for collaborating. (Khanzode et al, 2008). While these studies demonstrate the value which can

be derived from the visualization and teaming aspects created in interactive workspaces, it is still not clear which

traits of the systems are adding value, the value is considered in the context of meetings and not for performance of

differing tasks.

3. THE IMMERSIVE CONSTRUCTION LAB

The interactive workspace utilized for this study was the Immersive Construction (ICon) Lab at Penn State. The

ICon Lab was developed in stages starting with a single rear-projected screen. Currently, the ICon Lab has three 6

feet (1.8m) tall by 8 feet (2.4m) wide screens, with the two side screens angled at 30 degrees from parallel from the

center screen as shown in Figure 1. All three screens are rear projected, with two projectors for each to allow for

passive stereo visualization. The display is run from a single Windows-based desktop computer.

In addition to the central display, a single interactive whiteboard is set up adjacent to the left side of the main

display. The interactive whiteboard is connected to a separate computer from the main display. Along with the

interactive whiteboard a set of 20 tablet PC’s are housed within the ICon Lab. The tablets are set up to allow the

users to wirelessly push the screen image to one of the three screens of the central display.

72


Figure 1: Images of teams using different lab configurations and a rendering of the Immersive Construction Lab at

Penn State.

4. RESEARCH METHODOLOGY

The undertaken analysis is an intermediate step in an ongoing study to validate the INVOLVE framework for

utilizing interactive workspaces more effectively. The validation process utilized a series of observational quasi-

experiments for different tasks in conjunction with questionnaires to gain insight into the participants’ perspectives

of the media use for the four different types of tasks identified for use with the INVOLVE Framework: Create,

Integrate, Examine, and Focus. The 4D review task was chosen as a means of evaluating the interaction and display

parameters influencing an examination task. In this analysis, an undergraduate class in architectural engineering at

Penn State performed a 4D review activity in the ICon lab as part of a semester long project. An examine task is

defined as: Reviewing the design/solution for viability and to ensure compliance with the project goals, code

requirements, and owner and design intent (Leicht, 2009). The selected media elements were chosen as part of the

ongoing framework validation to help identify how the interaction with the 3D model and the use of shared

interaction with the display might impact the collaboration amongst the teams.

The project chosen for the 4D review was part of a team project, the student teams performed estimates, scheduling,

and site utilization planning for the Long and Foster Headquarters building, located in Chantilly, VA. Of the 95

students in the class, 16 groups of volunteers were used with typically 3 members each. The class was very

homogeneous, with the class admission limited to students in architectural engineering with 6th semester standing,

the student body is aged 20-22 years old for over 95% of the students, approximately 80% male, with 35% having

some limited design or construction experience. The 4D review task was held during the 12th week of a 16 week

semester. The task for the students while in the lab was to review the 4D simulation and identify schedule conflicts.

The 16 groups were randomly assigned to one of four different arrangements within the lab. The arrangements were

set up around 2 central variables, the number of computers for interaction and navigability of the 4D model.

When the students arrived for the activity they were provided with a brief overview of the lab layout and the tools

available, then they were then given 45 minutes to review the 4D simulation. The students then submitted both

individual and a team list of issues. At the end of the period, the students were asked to rate 20 statements about the

utility and value of the lab on a 1-5 Likert Scale. The teams were asked to perform the task in the lab as an

opportunity for extra credit, with the option of submitting the questionnaire. The students had an incentive to

perform well with the results on the conflicts identified related to the extra credit they received, but the professor

and teaching assistant were not present for the activities and did not see the questionnaire results until after grades

were submitted for students.

Using the INVOLVE framework, the teams were divided into four treatments with four teams randomly chosen for

each of the treatments. The 4D model was developed with a list of known errors, potential areas for improvement,

and potential safety concerns. The model was exported into a video file, with eight groups provided with the

Navisworks 4D model and the other eight having the exported video, thus creating the variable for the interaction

with the 3D model. In regard to the display interaction, eight groups were provided with a central display with a

73


mouse and keyboard for operating the simulation, the other eight teams were provided with three laptops with the

displays shared on three large screens.

As shown in Table 1, there were four different configurations of the ICon Lab based on the traits of the interactive

workspace. In these configurations, the differing displays influenced the availability of interfaces for team members

to draw upon, while the software interaction was based on the availability of navigating the 3D model while

reviewing the 4D simulations. Layout III should prove the best layout for the task – with a shared central display and

greater interaction with the 4D model.

Table 1: Matrix showing the breakdown of the four layouts used for the study.

While the large class does provide a large sample for study, by breaking the 16 groups into 4 configurations, the

subsets for each layout are not large. Also, because the feedback is based on the questionnaire responses the results

are based on subjective evaluations of the team experience, and may be biased. Particular bias may result from the

students having limited exposure the ICon Lab creating a small novelty effect, though that should hold true for the

whole sample. Also, since the participants were all students there was little to no construction experience which

means the process and discussions were less efficient and there may have been an added level of difficulty for the

task when putting the students into a new environment to perform the review.

5. RESULTS AND DISCUSSION

There were four areas on which feedback was collected from the participants. With the small sample sizes for the

four different layouts, little was expected in terms of statistical significance. The intent was to identify if the

perceptions of the value for the displays aligned with the expectations for the variables. The evaluation of the value

of the displays was handled through observational studies of the group which are not reported within here.

To begin the questionnaire regarding the experience of performing the 4D review task in the ICon lab, the focus was

on overall feedback. The teams consistently enjoyed the experience with all four Layout average close to 4.0 out of

5.0 in agreement. The second question asked about the suitability of the ICon lab for the 4D review task. While all

74


of the layouts averaged above 4.0, in agreement that it was well suited, the disparity among the four layouts align

with the expected variables which contribute to the use of Interactive Workspaces for such a task. The layouts

which had the strongest agreement were the teams which had a central display and use of the model for navigation

and changing viewpoints. For the last two questions there was also general agreement among all of the layouts that

the individual members were involved in the process and that they would be likely to use the lab or a similar space

for this type of task in the future.

Figure 2: General feedback from groups regarding their experience using the ICon Lab for a 4D review

task. *The statements listed on the X-axis have a scale of 1 (Strongly Disagree) to 5 (Strongly Agree) on the Y-axis.

5.1 Feedback regarding the display interaction

After reviewing general items about how the team worked, the questionnaire considered the interaction with the

display amongst the four layouts. The first question focused on the value of the large display system. The teams

which used only the large display found it very useful with an average above 4.0 in agreement. The teams which

had laptops were just above neutral, with one participant explaining:

“I tended to look at the computer screen instead of looking at the big screens. I feel that using the

bigger screen is better but because I had to use the mouse to start and stop the simulation it was

just natural for me to look at the smaller computer screen.”

The second question suggested that there was little value to having multiple screens. The average across the four

layouts suggests a slight preference for multiple screens, but not much. The students using the lab Layout which

used the large display and the video agreed slightly with the comment that the extra screens were unnecessary. The

last two statements that the environment allowed the team to communicate effectively and for all members to

contribute were consistently agreed to above a 4.0 level by all layouts.

75


Figure 3: Feedback regarding the value of the display system. *The statements listed on the X-axis have a scale of 1 (Strongly Disagree) to 5 (Strongly Agree) on the Y-axis.

5.2 Model Interaction Feedback

When asked about the value of the interaction available when viewing the 4D simulation, the responses became

more disparate. The teams which had open access to navigate the model felt that they were able to view the

necessary perspectives within the model, with agreement above 4.0 for both layouts which had the model. The

layouts which had the video file of the simulation slightly disagreed, with both averaging below a 3.0. When asked

if the teams shared control of the model and display, the teams with laptops all agreed above a 4.0 level, the teams

with the model and central display agreed somewhat (3.5) and the teams with the video file and central display came

out at a neutral level – indicating a notable disparity in the perception of the groups about their ability to use the

virtual prototype. All four layouts agreed that the model was useful for communicating information about the

schedule issues above a 4.0 level. When asked if there was a single primary user of the model and display, again

Layout IV was neutral, and Layout II felt that there was no primary user, with Layout I and III close to neutral but

suggesting that some teams had a primary user.

76


Figure 4: Feedback regarding the value of model interaction. *The statements listed on the X-axis and a scale of 1 (Strongly Disagree) to 5 (Strongly Agree) on the Y-axis.

5.3 Collaboration Feedback

The last area for feedback was focused on the impact of the layout on the collaborative effort of the team.

Team members in all four layouts felt that the teams worked productively during the task, with an average

score above a 4.0 level, and most reaching 4.5. When asked if their team collaborated better than their

project team, most teams were neutral or slightly above. Team members in all layouts felt that everyone in

the group contributed ideas and suggestions during the task with an average score above a 4.0 level. When

asked if it was challenging to reach a consensus on the issues in the schedule, the layout responses varied

by notable amounts. Layout I and IV felt that it was relatively easy to reach a consensus, while Layout II

and Layout III closer to being neutral, again creating a disparity by layouts.

Figure 5: Feedback regarding the collaboration which took place in each team. *The statements listed on the X-axis have a scale of 1 (Strongly Disagree) to 5 (Strongly Agree) on the Y-axis.

What stands out is how closely the responses regarding meeting a consensus reflect the responses about how well

suited the lab was for the 4D review task. Layout III had the highest rating for the lab being suitable for 4D review

and found reaching consensus the easiest among the four layouts. As suggested earlier, Layout III was identified as

77


ideal for the task. Reaching a consensus implies getting the team members to understand the issue and all agree that

it is a problem; both the individual student perception feedback and the quantitative evaluations of the outcomes

show that this configuration was the best for teams to reach consensus for this type of task.

6. CONCLUSIONS AND FUTURE WORK

This paper presented questionnaire results for teams utilizing an interactive workspace for a 4D review task. The

use of the Interactive Workspace for the 4D schedule review task was planned to focus upon the shared interaction,

the shared display, and the software capabilities for manipulating the virtual prototype. The mouse and keyboard

interface were the chosen forms of interaction, with the availability of separate laptops allowing select teams to

review the schedule individually first. The assumption of Layout III as the best layout was encouraged by the

feedback as the highest rating for suitability for the task and for ease of reaching consensus.

The outcome from the 4D review suggests that the choice of the interaction and the display relate to the ability of the

team members to communicate for a task. The value in planning the media for this example can be seen in the

feedback from the participants suggesting that the media indicated when using the INVOLVE Framework aligned

with the intended results for an Examine Task based on the perceptions of the participants. In addition, the teams

found the experience in the lab more enjoyable than typical meetings and would again use the interactive workspace

for performing similar tasks in the future.

Future work in this area will explore more thorough analysis of the displayed value of the systems when

collaborating through observational studies of these tasks. The study will also explore other tasks that may find

value in utilizing interactive workspaces. Also, studies of the value of different forms of interaction depending on

the task at hand and interests of the team members involved.

7. ACKNOWLEDGEMENTS

We would like to thank the student participants for their involvement during the study presented in this paper. We

also thank the National Science Foundation for their support of this research through Grant No. 0348457 and

0342861. Any opinions, findings, conclusions, or recommendations are those of the authors and do not reflect those

of the National Science Foundation or the project participants.

8. REFERENCES Air Force (1994). Glossary, Proceedings of the NSIA Spacecast 2020 Symposium, 9-10 November 1994,

Washington, DC, USA. http://www.fas.org/spp/military/docops/usaf/2020/app-v.htm

Fruchter, R. (2005). Degrees of engagement in interactive workspaces, AI & Society, 19, pp. 8-21.

Gopinath, R., and Messner, J. I. (2004). "Applying immersive virtual facility prototyping in the AEC industry."

Proceedings of the 4th International Conference of Construction Applications of Virtual Reality, Lisbon,

Portugal, Sept. 14-15, 79-86.

Issa, M., Rankin, J., Christian, J., and Pemberton, E. (2007). Using interactive workspaces for team design project

meetings. CONVR 2007: 7th Conference of Construction Applications of Virtual Reality, State College, PA,

USA, Oct. 22-28.

Khanzode A., Fischer M., and Reed D. (2008). Benefits and lessons learned of implementing building virtual design

and construction technologies for coordination of mechanical, electrical, and plumbing systems on a large

healthcare project, Journal of Information Technology in Construction, Vol. 13, Special Issue: Case studies of

BIM use, pp. 324-342.

Leicht, R. M. (2009). A framework for planning effective collaboration using interactive workspaces. Ph.D. Thesis,

The Pennsylvania State University, University Park, PA, USA.

Leicht, R., Maldovan, K. and Messner, J. (2007). A framework to analyze the effectiveness of team interactions in

virtual environments, Proceedings of the 7th International Conference on Construction Applications of

Virtual Reality, University Park, PA, USA, October 22-23.

Liston K., Fischer M., and Winograd T. (2001). Focused sharing of information for multi-disciplinary decision

making by project teams. Journal of Information Technology in Construction, 6, pp. 69-82.

Wang L., Messner J., and Leicht R. (2007) – Assessment of 4D modeling for schedule visualization in construction

engineering education, Proceedings of 24th CIB w78 conference on information technology in construction,

June 26-29, 2007. Maribor, Slovenia.

78


DESIGN SCENARIOS: METHODOLOGY FOR REQUIREMENTS DRIVEN PARAMETRIC MODELING OF HIGH-RISES

Victor Gane, PhD Candidate,

CIFE, Stanford University;

[email protected]; http://stanford.edu/~vgane

John Haymaker, PhD, AIA, LEED AP, Assistant Professor

CIFE, Stanford University;

[email protected]; http://stanford.edu/~haymaker

ABSTRACT:

This paper introduces a collaborative, parametric performance-based design methodology that enables teams to

systematically generate and analyze high-rise building design spaces based on multi-stakeholder requirements.

Building design involves investigating multidisciplinary design spaces with a high number of project-specific

variables and constraints. In practice at leading architecture firms today, conceptual design methods support

generating very few options that respond to a limited number of design requirements. As a result, potentially better

performing design solutions are overlooked. Our research synthesizes a novel, collaborative design methodology

called Design Scenarios (DS). The methodology consists of five process steps: (1) Requirements Model used by a

multidisciplinary team to collect, weigh and prioritize multi-stakeholder requirements, (2) Design Strategy used to

formally transform into parametric models the identified requirements by proposing potential enabling design

parameters and identifying conflicting and enabling relationships amongst requirements and design parameters, (3)

Parametric Process Model used to generate, manage and communicate the complex structure of a resultant

parametric product model from these relationships; (4) Parametric Model used to generate design spaces

responsive to identified requirements, (5) Decisions Model used to support the consensus-building and

documentation of the best decision by visually reporting the design options' performance back to the designers and

stakeholders. We applied DS on a case study presented in this paper. The research is unique in its development of a

method to formally generate parametric models from requirements, and for its industrial-scale, practice-based

integration and testing of formal design and decision making methodologies for high-rise building design.

Improvements are anticipated both in the quality of the design process by reducing uncertainty and inefficiency, and

in the resulting product by enabling more options to be considered from more perspectives.

KEYWORDS: Design space, parametric modelling, process modelling, requirements engineering.

1. OBSERVED PROBLEM The market economy requires project teams to design quickly and cheaply; however, research shows that successful design is largely a function of clear definition of end-user requirements (Rolland, 2005) and the generation and multidisciplinary analyses of a large quantity of options (Kelley 2006). Every project comes up against an inevitable tension between design exploration and process efficiency. Take high-rise design for example. We recently conducted a benchmarking survey of existing conceptual high-rise design practice to determine the performance of leading design teams. We found that on average a multidisciplinary team averaging 12 people can normally produce only 3 design options during a design process that lasts on average 5 weeks. Most of this time is spent by architects on generating and presenting a small number of design options. Little time is dedicated to establishing / understanding project goals and running multidisciplinary analysis. These analyses are inconsistent and primarily governed by architectural rather than multidisciplinary criteria (Gane & Haymaker 2008). Better performing designs are likely left undiscovered.

How can high rise building project teams improve design and critical thinking? Understanding and efficiently managing multidisciplinary requirements early in the design process is a major challenge. So is translating these requirements into a wide range of design options that designers can quickly analyze and systematically choose from. Several points of departure partially address these issues. Design Theory helps us understand the general process of design and define strategies to search the design space. Process modeling can help represent and measure goal-driven design processes. Requirements engineering can help design teams define and manage their building design

79


criteria in terms of formally structured goals and constraints. Parametric modeling can help efficiently generate geometric options. High-rise Design Methods help categorize the types of high-rises and elicit a list of design constraints, criteria and performance metrics that each category entails (we summarize these in Gane & Haymaker, 2008). Even with these theories and methods, our benchmarking study shows that the Architecture Engineering Construction (AEC) industry still lacks a methodology that enables project teams to efficiently integrate them into practice. They lack a methodology to define and prioritize requirements, translate these requirements into geometrically flexible parametric models, to analyze these models efficiently from multiple perspectives, and to understand the multidisciplinary tradeoffs of individual options and spaces of options. In another paper we describe how the lack of such a method substantially reduces the effectiveness of parametric methods and stalls multidisciplinary design and decision making processes (Gane & Haymaker 2007). This research establishes such a methodology and begins to test its impact in practice.

2. THEORETICAL POINTS OF DEPARTURE In this section we describe the fundamental points of departure for this research.

2.1 Design theory Design is a creative process, where part of the task is to formulate the problem itself (Simon 1969). Design teams are aided by multi-stakeholder value-based design and decision making methodologies (Lewis et al 2007, Jin & Danesh 2007, Keeney & von Winterfeldt 2007). AEC focused researchers are developing related theory and methodologies, describing the design as (1) identifying a set of requirements; (2) prioritizing among these requirements; (3) developing preliminary solutions; (4) evaluating solutions; (5) establishing final design requirements, preferences and evaluation criteria (Akin 2001). Others are applying these concepts in formal design and decision making methodologies (Ellis et al 2006, Haymaker & Chachere 2007). While designing, teams construct a design space, formulated as the sum of the problem space, solution space, and design process (Krishnamurti 2006). Two prevailing strategies emerge to describe the process of constructing a design space: breadth first, depth next or depth first, little breadth. Designers typically consider a very small number of alternatives as a result of cognitive limits (Woodbury and Burrow 2006). Therefore, they are forced to make decisions that are not optimal but only satisfactory according to a pre-set aspiration level. In contrast, expert designers prefer the breadth first, depth next strategy (Akın 2001). As a result, multiple alternatives help reveal new directions for further exploration that the designer wouldn’t have thought of otherwise. Design teams need dynamic rule-driven systems that help them set up and manage design generation processes with the right balance of breadth and depth strategies to best address the multidisciplinary requirements.

2.2 Process modelling Design theory helps us understand the general process of design; however it does not help us determine how specifically to represent and measure design processes. Such understanding can help quantify and compare the performance of existing and proposed processes, as well as provide the tools that help organizations adopt the proposed processes. A widely accepted implementation method is process modeling. Multiple process models for AEC have been proposed (Froese 1996). Among other significant process models are IDEF0 (Integrated Definition Methods) used to model decisions, actions, and activities of an organization or system, the Narratives (Haymaker et. al. 2004) that provide a means to model information and the sources, nature, and status of the dependencies between information, and Value Stream Mapping (Tapping & Shuket 2002) used to illustrate and analyze the flow of actors, activities, and information that produce value in a given process in order to assist in process re-engineering. Despite the wealth of existing process modeling methods, an important need specific to this research is not adequately met – a representation formalism for communicating the structure of parametric models and their multiple levels of multidisciplinary information dependencies.

2.3 Requirements engineering Poor definition or misunderstandings of requirements are major causes of system failure in software engineering (Rolland & Salinesi 2005), mechanical engineering (Hsu & Woon 1998), and in AEC (Kiviniemi et. al. 2004). Systematic methods to screen and prioritize among design requirements have been proposed (i.e. Quality Function Deployment (Takai & Ishii 2006, Leary & Burvill 2007), PREMISS (Kiviniemi et. al. 2004), MACDADI (Haymaker & Chachere 2007). While some methods help designers translate requirements into feasible design

80


options (i.e. Chen & Pai 2005), no systematic method exists for reliably generating parametric design spaces from a multidisciplinary requirements model. Requirements Engineering (RE) can help formalize such a process. RE is used as a means to overcome the drawback of traditional software development methods, in which the developed systems are often technically good but unable to appropriately respond to the users’ needs (Rolland & Salinesi 2005). RE helps determine the features the system needs to satisfy, and how the system is to be constructed (Ross and Schoman 1977). Reasoning with requirements can also help resolve conflicts among stakeholders. To develop better solutions designers need to understand how requirements relate to each other as well as to other elements in the requirements model. AND/OR graphs are used to capture goal refinement links (Lamsweerde 2000). While RE provides an actionable method to help designers translate requirements into better solutions, it does not provide a specific means to translate design requirements into options generated with parametric methods – a task that this research proposes to address.

2.4 Parametric modelling Parametric computer-aided design (CAD) is a design methodology used to create design spaces and manage geometric dependencies within a model. The concept of “features” not present in other CAD systems, encapsulates generic shapes or characteristics of a product with which designers can associate certain attributes and knowledge useful for reasoning about that product (Shah & Mäntylä 1995). Using parametric CAD tools designers can create an infinite number of objects, geometric manifestations of a previously articulated schema of variable dimensional, relational or operative dependencies (Koralevic 2003). However, designing with multiple constraints without an efficient constraints management system is a daunting task. An example of a constraint management methodology are the design sheets, in which design models are represented as constraints between variables in the form of nonlinear algebraic equations organized into bipartite graphs and constraint networks (Reddy et al 1996). Using only design sheets to define high-rise parametric models would be challenging given the overwhelming number of constraints that need to be described at the schema level and the inability to visualize geometry. As a result, in parametric systems, Geometric Constraint Programming (GCP) is used to graphically impose geometric constraints to solve the relevant nonlinear equations without the user explicitly formulating them (Kinzel et al 2007). This research, however, identified the lack of a formal method to determine constraints and parameters for constructing parametric models with sufficient flexibility to respond to a set of multidisciplinary requirements.

2.5 High-rise design methods This point of departure helps categorize the types of high-rises and elicit a list of design constraints, criteria and performance metrics that each category entails. In prior work we have reviewed many specific goals and methods for high-rise design found in literature (Gane & Haymaker, 2008). Understanding and translating these requirements into quantifiable architectural and energy performance goals and constraint is important in the process of building parametric models.

3. RESEACH QUESTIONS Our research proposes to answer the following questions: • What is a method to generate conceptual parametric models of high-rise design spaces that respond both to

multidisciplinary performance requirements? • What is a method to synthesize these requirements models, parametric models, performance analysis models,

and decision making models into an effective and efficient methodology for high-rise design practice?

4. RESEACH METHOD DESCRIPTION In Figure 1 illustrates the updated conceptual design process called Design Scenarios that we designed to address these questions. Such a process starts with a clear set of architectural and engineering performance requirements that help establish the design space. The identified requirements guide the generation of design alternatives within the established design space. Alternatives are then formally analyzed in discipline specific tools. A bi-directional relationship between alternatives and analyses supports a recurring refinement process. The multi-attribute performance of each alternative helps determine its value and establish a formal decision making process. Analyzed options are then correlated with the requirements to determine their value and choose best option.

The goal of this research is to: help design teams formally identify architectural and engineering performance requirements and translate them into parameters for developing design alternatives with parametric CAD (link between boxes 4.1 & 4.2, 4.2 & 4.3, Fig. 1); help determine the value of each analyzed option in relation to specific requirements (link between boxes 4.1 & 4.4). Parallel research is being conducted at CIFE, Stanford to automate the

81


engineering performance (i.e. energy, daylight, thermal comfort) analyses generated with parametric CAD (link between Model based analyses & box 4.4), as well as determine importance of design parameters in relation to specific requirements (link between Model based analyses & box 4.3).

FIG. 1: Proposed conceptual design process.

Following is a description of the models comprising the Design Scenarios methodology.

4.1 Requirements Model (RM) The augmented conceptual design process starts with building a RM. Unlike current practice, in which the architect unilaterally makes most of the early design decisions based on a set of loosely defined requirements, building a RM will require in addition to stakeholders the participation of all design disciplines (architect, structural, and mechanical engineers). The RM is hosted online to facilitate remote definition of a comprehensive set of high-level project requirements by the project users (stakeholders and design team), who prioritize them according to their level of importance. Each participant has to distribute 100 percentage points to each identified goal, which represents their preference. Inputs range from stakeholder defined requirements (i.e. architectural brief, budget, building efficiency) to those established by the multidisciplinary design team (i.e. preferred design language, daylight factor, energy comfort). The output of this model is a stacked column chart distinguishing the stakeholders’ and design team’s priorities and constraints. Determining a comprehensive set of multidisciplinary requirements that helps eliminate the non productive ambiguity in current early decision making practice is the major benefit of building a RM. The RM also provides the formal value function for evaluating design options.

4.2 Design Scenario Model (DS) Building a DS will help the design team map requirements to design strategies (i.e. determining means of achieving requirements). The goals and constraints from the RM are grouped and serve as the initial inputs into the Design Scenario environment. A DS consists of five levels of hierarchically built information – (1) the RM-established high-level requirements; (2) Action items; (3) Strategies; (4) Parameters; (5) Parametric constraints. • Requirements - Building on concepts defined in the RM, the DS further categorizes and defines the

requirements in terms such as : Quantifiable (i.e. maximize use of daylight in 50% of interior space, provide 50,000 sq m of usable area) and Non-Quantifiable (i.e. use a specific design language). Requirements are further decomposed into: Goals (objectives – i.e. design within $50m budget) and Constraints (requirements whose satisfaction is compulsory – i.e. 50,000 m2 usable area).

• Actions items - Discipline-specific design team leaders decompose each relevant high-level requirement into Action Items. These are determined by asking the HOW question for each requirement. Building on concepts from Artificial Intelligence (Lamsweerde, 2001), the DS describes relationships between Action Items through AND/OR links. For example, to provide daylight in 50% of the building interior designers need to: (a) control the building orientation; AND (b) control the lease span; OR (c) introduce shading fins; OR (d) introduce light shelves; AND (e) control window configuration; AND (e) control glass type, etc. All Action Items with AND links are required to satisfy the original requirement, whereas an OR link illustrates a choice of action;

• Strategies – In case Action items cannot be directly translated into geometric or material parameters (i.e. choose window configuration), design team leaders further decompose these into Strategies (i.e. butt glazed, expressed mullions, unitized panel, etc).

• Parameters - Action Items or Strategies are decomposed into geometric and/or material parameters, the value of which will determine the design’s performance in relation to a specific requirement. For example, to

introduce shading fins a designer must create a depth parameter of length type AND an inclination parameter of angle type after anticipating the need to adjust the shading fins’ geometry in response to the provide daylight

82


in 50% of interior quantifiable goal. Parametrically controlling the depth and inclination of shading fins offers efficient means to refine the geometry after a formal daylight analysis is performed.

• Parametric constraints – When identified parameters are required to be within specific ranges, these parameters are decomposed into Parametric constraints (i.e. No. Floors parameter decomposed into Lower limit

– 30 floors; Upper limit – 50 floors parametric constraints). The DS will also allow design teams to model relationships between the above concepts. The explicit definition of parameters will help determine potential interdependencies. For example, the shading fins depth input parameter will impact the shading fin area output parameter, which in turn will determine the shading fin cost parameter used in calculating the overall design cost. Potential conflicts among requirements may not become apparent just by developing a Requirements Model, but can be identified when Action Items and Parameters are determined and related. For example, introduce shading fins Action Item can potentially conflict with design within $50m budget quantifiable goal given the additional cost of external fins. A DS will explicitly show such conflicts and help the design team mediate an updated set of requirements with the stakeholders, thus avoiding costly design revisions common in current practice. Knowledge of these dependencies will guide the CAD specialists (i.e. parametric modelers) in creating a model that is optimally constructed to address the identified requirements. The DS model output is a bipartite graph whose orientation, level of detail, and format are user determined.

4.3 Parametric Process Model (PPM) PPMs help the design team illustrate and manage the logical construct and technical implementation of a DS in a parametric CAD model. CAD specialists build PPMs. Input, output and constrained parameters and the relationships/dependencies established in the DS serve as the initial inputs. A PPM also consists of Components made of geometric and construction elements, PowerComponents made of generic components grouped and intended to be used in unique contexts, Geometric Constraints used to establish relationships among geometric elements and parameters, Information Dependencies (i.e. Component A dependent on Component B or Component A dependent on input parameter(s), etc). A PPM is a formal roadmap to building a parametric model. Parameters and relationships established in the PPM are used as inputs to automate their generation in a chosen parametric modeler (i.e. Digital Project, Generative Components). The output of the process is a bipartite graph whose orientation and format is user determined and the beginning of a parametric model that requires the CAD specialists to build the components described in the PPM graph and link the automatically generated parameters.

4.4 Options Analysis Model (OAM) OAMs help the design team evaluate how each option generated from a parametric model and analyzed in discipline specific tools (i.e. daylight in Radiance) ranks in relation to the high level requirements identified in the RM. Scores measured in percentage points are assigned to each option based on low and high benchmarks (i.e. high benchmark – minimize cost to $80mln, low benchmark – minimize to $100mln). If an option achieves a goal, it receives 100% score. If it exceeds it (i.e. $70mln, it receives the percentage scored above the high benchmark – 112.5%, etc). This allows design teams to determine the impact of each option’s performance against the RM goals. Goals, however, ranked in terms of their importance to each discipline are also measured in percentage points. To determine the final value of each option, the impact score for each goal is multiplied with the appropriate goal importance score and summed into a final value function score. The outputs of OAM are spider diagrams and column graphs. An OAM offers design teams a formal unifying structure and communication tool for describing and managing the quantitative and qualitative analyses of options.

5. CASE STUDY – HIGH-RISE IN SAN FRANCISCO, CA

We are developing a web-based Design Scenarios software platform to significantly improve the DS modelling

process by partially automating the generation of each consecutive model and feed the generated parameters into a

parametric modeller. Prospective validation of DS in practice is expected in 2010-2011. Our research method (Hartmann et. al. 2008) involves using an embedded researcher who will spend approximately one month in an AE

firm. DS will be introduced in several training sessions. Studio members will have prior experience of using

parametric modelling on several projects. DS will be used by the trained team on several case studies (office towers), in which the conceptual design process lasts about three weeks. Our goal is to improve DS through iterative

implementation by testing it against the following metrics: (1) goal definition clarity; (2) concept design duration;

(3) No. generated options; (4) team size / composition; (5) total man hours per discipline; (6) time per task; and (7)

explicit analyses performed. We describe our validation of the Design Scenarios using retrospective data and a

83


hypothetical high-rise project in San Francisco. The illustrated models are abbreviations of larger models for the purpose of this paper.

5.1 Requirements Model (RM)

The project commenced with a meeting between the stakeholders (developer team) and the senior design team members (architect, mechanical and structural engineers). The designers’ objective was to elicit a comprehensive list of project requirements and help the developers determine target metrics for those requirements that traditionally fall outside of their domain of expertise (i.e. energy efficiency). The stakeholder-defined architectural brief was used to establish the initial requirements, most of which were determined to be constraints (i.e. area requirement, height limitation). Several quantifiable and non-quantifiable goals were also identified (i.e. maximizing building efficiency to 85%, minimizing construction cost based on the project budget of $80m, design to be widely recognized in San Francisco). The benefit of this new meeting format became apparent in the subsequent discussion, when the senior mechanical engineer (normally not present in the first few meetings) suggested several additional goals for improving the building’s energy efficiency and supported by target metrics (i.e. maximize the use of daylight to 500lux – optimal for office spaces; maximize thermal comfort to a range between 22-260C; minimize energy consumption to 600MJ/m2/year (~1000MJ/m2/year is the current average). Once the list of goals was accepted by all parties, every participant individually ranked each goal according to his/her preference, measured in percentage points. This helped determine the most important goals and the weighting preferences for each discipline. For example, the developer team, the architect and the structural engineer saw the recognizable design as their leading requirement where as the mechanical engineer gave more weight to daylight, thermal comfort and energy

consumption goals. The outcome of the meeting was a good understanding of what the project requirements were and their level of importance to each discipline. These were formally represented in the Constraints and Goals

models (Fig. 2).

FIG. 2 – a) Project constraints whose satisfaction was mandatory; b), c) Ranked project goals - participants had to

distribute a percentage of preference (totalling 100%) to each identified goal

5.2 Design Scenario Model (DS)

Back in the studio the senior design team organized another meeting, in which the high-level requirements from the RM served as the starting point for building the DS. The meeting lasted one day, during which the constraints and goals were first grouped and decomposed into Action Items (AI). The process was collaborative given that some constraints and goals resulted in multiple AIs suggested by one or several disciplines. For example, two AIs were proposed for Recognizable design goal - creating an aesthetically unprecedented design (architect) and a structurally

unprecedented design in San Francisco (structural engineer). These AIs were acknowledged as vital to the project given the importance level of the parent goal to the developer and the design teams. Both AIs were assigned an AND Link and therefore required to be implemented in the final design. In case of multiple OR Links, one option must be selected. Similarly, the mechanical engineer developed AIs for his relevant goals. For example, for the maximize daylighting goal he proposed three AIs with appropriate goal refinement links (control building orientation

84


FIG. 3 – Case study project Design Scenario Model (DS)

85


– AND Link; introduce lightshelves – OR Link; choose glass type according to orientation – OR Link). The architect proposed additional three for the same goal (choose window configuration – AND Link; control the lease span – AND Link; introduce central atrium – OR Link). The architect assigned an OR Link to the central atrium AI because of his knowledge of how important minimizing construction cost and maximizing efficiency were to the developer. This understanding helped identify a potential goal conflict. An atrium in a project constrained by height and site setback made it impossible to achieve the efficiency and potentially the construction cost goals. Design Scenarios, however, is built as a recursive process. Therefore, the atrium option was kept in the design scenario in case the preferences for high-level goals changed.

Some AIs required further decomposition into Strategies. For example, after evaluating the site and its context, the architect suggested three possible strategies for the aesthetically unprecedented design AI – round OR rectilinear

footprint, AND mainly glass exterior. Once all AIs and strategies were finished, each discipline proceeded to decomposing these into input and output parameters. For example, for the Tower height range constraint the architect proposed one AI – determine No. floors, which was decomposed into Total No. floors AND Floor height input parameters. When constraints were specified, parameters were further decomposed into Parametric constraints (i.e. Total No floors -> Lower limit – 30, Upper limit – 50). When appropriate, parameter interdependencies were specified for determining output parameters (i.e. Floor height will determine the Window height output parameter).

Without formal knowledge of parametric modelling, the senior design team was able to describe Action Items in terms of parameters and parameter interdependencies. The abbreviated DS model is illustrated in Fig. 3.

5.3 Parametric Process Model (PPM)

Fig. 4 illustrates the case study PPM. Each component that has a visual preview is numbered. The input variables and geometric constraints for each component are lettered and their location is shown in the component preview.

The completed DS model served as a starting point for parametric CAD specialists to build the PPM. The AIs helped determine the goals and constraints that affected the initial decisions of how to build a parametric model. Input and output parameters were correlated with the appropriate AIs before building PPM components. For example, the senior architect specified two geometric possibilities for the aesthetically unprecedented design AI - curved or rectilinear footprint, which required the parametric model to support changes in geometric topology. Therefore, the Ground footprint (1) (component (1), Fig. 4) was composed of BSpline of order 2 that supported such transformation. Being a linear BSpline it can either be a line or a curve depending on how it is geometrically constrained to other geometric elements. The choice of geometry helped determine the geometric constraints controlling the component and assign the appropriate parameters established in the DS. For example, building side

length (B) input parameter controls the length between the BSpline endpoints, which use a concentric (E) and tangency (F) constraint to a skeleton of construction (dashed) lines. Tangentially constraining the BSpline endpoints to the construction lines will change the geometry’s topology. The construction lines’ endpoints are coincidentally

(C) constrained to establish a pin connection. The lines use a perpendicular constraint (D) to avoid arbitrary rotation. In response to control building orientation AI the rotation angle (A) parameter is introduced by constraining the angle between the construction line end point and the chosen axis of the user coordinate system.

Having established the ground footprint allows the dependent components to be constructed. For example, Building

core footprint (2) is constrained through length to exterior wall (A) parameter to the ground footprint (1) BSplines, which established a component dependency. Changes in the footprint impacted the core unless the length to exterior wall parameter was adjusted to compensate the increase or decrease in the building length parameter. The remaining components were similarly built.

86


FIG. 4 – Case study project Parametric Process Map (PPM) used for technical implementation of the DS in a

parametric model, managing the model and communicating its construct.

5.4 Options Analysis Model (OAM)

The design team used the resultant parametric model to generate over 1,000 design variations in one week by operating the values of several key input parameters (i.e. building side length, lease span, floor height, lightshelves depth, window width). Building an OAM required them to analyze the options to formally understand their performance. However, most of the options were discarded after not meeting constraints that were defined as output parameters (i.e. No. floors, building net area, and building efficiency). The options that passed these requirements were visually analyzed by the senior designers, who chose 20 options (5 shown in this paper) for further formal analysis outside of the parametric model.

For example, to calculate the daylight performance, mechanical engineers used Autodesk Ecotect. Key parameters that determined the performance for each option were Lease span, Floor height, Light Shelf depth, and Window

width. The analyses results were compared against the target value of 500lux. However, no formal model based analysis was performed for thermal comfort and energy consumption. Currently, a widely accepted tool for performing such analysis is EnergyPlus by the US Department of Energy. Interoperability and model preparation issues make the use of this tool in early stages of design daunting. As a result, the mechanical engineers evaluated the thermal comfort and energy consumption performance of the selected options by analyzing a combination of geometric parameters used to generate each option. For the Minimize energy consumption goal, for example, each option was evaluated in terms of four key parameters: Lease span, Floor height, Lightshelves depth, Window width. A deeper lease span, a taller floor height, and a smaller light shelf (used to block direct sunlight along the building perimeter) meant greater volume to condition with mechanical systems and greater variation in internal temperature, which made the option less energy efficient.

87


The OAM required building the impact and value models of each analyzed option. After performing each analysis on all selected options, the senior designers determined the low benchmark for each discipline specific goal and then assigned a score representing the performance measured in percentage points. For example, any option that resulted in an average daylight value of 500lux scored 100% (benchmark determined in the Requirements Model), and 0% if it had an average of 0lux. The impact score for each option was determined by summing the scores of each goal for that option (i.e. Option 1 = 515 - Fig. 5a). However, as determined in the RM, goals were ranked according to their level of importance to the stakeholders and the design team (i.e. Recognizable design = 115 points – Fig. 2b). The OAM required determining the option total value score before choosing a winning design. The sum of each goal in the RM was translated into a percentage of importance from the total of 400 points that the four participants had to distribute (i.e. Recognizable design goal scored 115 points or 29% overall importance – Fig. 2c). Multiplying each goal’s preference score to the impact score determined the goal value of each option (i.e. Option 1, Recognizable design – 29% x 70 = 20). Summing the value of all goals resulted in the overall value per option (i.e. Option 1 = 86 - Fig. 5b, 5c) Fig. 5d illustrates the parameter values determining the 5 selected options and total value scores.

a) b)

c) d)

FIG. 5 – a), c) Case study OAM impact model; b), d) value model used for final decision making – option 1 scored

the highest and was considered the winning design.

6. CONCLUSIONS

In previous work (Gane & Haymaker 2008) we have benchmarked current conceptual high-rise design processes in terms of the metrics listed in Fig. 6. This paper presented a new collaborative design methodology called Design Scenarios and illustrates a case study of applying DS on a high-rise project. We compare the metrics of current practice with anticipated metrics from the case study. We wish investigate how a high degree of goal definition clarity can help a multidisciplinary design team build parametric models and explore and analyze a much larger segment of the design space in less time. Unfortunatley the case study method of implementation of DS willmake it difficult to claim generality in our findings.the Design Scenarios methodology and defined metrics can guide research and development efforts to improve these measurements, and can serve as a benchmark for comparing new design methods, tools, and processes.

88


FIG. 6 – Comparison between metrics describing current conceptual design performance and the case study results.

Many more design options can be generated and analyzed with Design Scenarios in less time.

7. REFERENCES Akın, Ö. (2001). “Variants of design cognition”. Design Knowing and Learning: Cognition in Design Education.

Eastman, C., Newstetter, W., & McCracken, M., Eds., pp. 105–124. New York: Elsevier. Chen, D., Pai, W. (2005), “A Methodology for Conceptual Design of Mechanisms by Parsing Design

Specifications”. Journal of Mechanical Engineering, Vol. 127, Issue 6, pp. 1039-1045. Ellis, P., Griffith, B., Long, N., Torcellini, P., Crawley. D., “Automated Multivariate Optimization Tool for Energy

Analysis”, Proceedings of SimBuild 2006, MIT, Cambridge, Massachusetts. IBPSA-USA. Froese, T. (1996). “Models of Construction Process Information”. Journal of Computing in Civil

Engineering. pp. 183-193. Gane, V., Haymaker, J., (2007). “Conceptual Design of High-rises with Parameteric Methods”. Predicting the

Future, 25th eCAADe Conference Proceedings, ISBN 978-0-9541183-6-5 Frankfurt, Germany, pp 293-301. Gane, V., Haymaker, J., (2008). “Benchmarking Current Conceptual High-rise Design Processes”. Submitted to

ASCE Journal of Architectural Engineering. Haymaker, J., Chachere, J., (2007). “Coordinating goals, preferences, options, and analyses for the Stanford Living

Laboratory feasibility study”. Conference paper. Hsu, W., Woon, I. M. 1998. “Current Research in the Conceptual Design of Mechanical Products”. Computer-Aided

Design, 30, 1998, pp 377-389. Jin, Y., Danesh, M. (2007). “Value Aggregation for Collaborative Design Decision Making.” Keeney R., von Winterfeldt, D. (2007). “Practical Value Models.” W. Edwards, R. Miles, and D. von Winterfeldt

(eds.) Advances in Decision Analysis New York: Cambridge Kinzel, E., Schmiedeler, J., Pennock, G. (2007). “Function Generation With Finitely Separated Precision Point Using Geometric Constraint Programming”. Journal of Mechanical Engineering, Vol. 129, Issue 11, pp.

1185-1191. Kiviniemi, A., Fischer, M., Bazjanac, V., Paulson, B. (2004). “PREMISS - Requirements Management Interface to

Building Product Models: Problem Definition and Research Issues”. CIFE working paper #92

Kolarevic, B. (ed), (2003). “Architecture in the Digital Age: Design and Manufacturing”. Taylor & Francis. Kelley, D. “Design Thinking”. Accessed at http://www.extrememediastudies.org/extreme_media/

1_navigating/pdf/navigating_design_thinking.pdf Krishnamurti, R. (2006). “Explicit design space?” Artificial Intelligence for Engineering Design, Analysis and

Manufacturing. 20, 95-103. Lamsweerde A (2000). “Requirements engineering in the year 2000: A research perspective”. Proceedings of 22nd

International Conference on Software Engineering, (ICSE’2000): Limerick, Ireland, ACM Press, pp. 5–19. Leary, M., Burvill, C. (2007). Journal of Mechanical Engineering, Vol. 129, Issue 7, pp. 701-709. Lewis, K. E., W. Chen, and L. Schmidt (eds.) (2007). “Decision Making in Engineering Design”. New York: ASME

Press

Reddy, S., Fertig, K., Smith, D. (1996). “Constraint Management Methodology for Conceptual Design Tradeoff Studies”. Proceedings of the 1996 ASME Design Engineering Technical Conferences and Computers in

Engineering Conference. Aug. 18-22, 1996. Irvine, California. Rolland, C., Salinesi, C. (2005). “Modeling Goals and Reasoning with Them”. Engineering and Managing Software

Requirements.

89


Ross, D., Schoman, E. (1977). “Structured Analysis for Requirements Definition”. IEEE Transactions on Software

Engineering, Vol. SE-3, No. 1. Simon, A.H. (1969), “The sciences of the artificial”, MIT Press Cambridge MA. USA. Shah, J., Mäntylä, M. (1995). “Parametric and Feature-Based CAD/CAM: Concepts, Techniques, and

Applications”. Wiley, John & Sons, Inc. Takai, S., Ishii, K. (2006). “Integrating Target Costing Into Perception-Based Concept Evaluation of Complex and Large-Scale Systems Using Simultaneous Decomposed QFD”. Journal of Mechanical Engineering, V 128, Issue 6,

pp. 1186-1196.

Tapping, D., Shuker, T. (2002). “Value Stream Management”. Productivity Press. Woodbury, R., Burrow, A., (2006). “Whither Design Space?” Artificial Intelligence for Engineering

Design, Analysis and Manufacturing, 20, 63-82

90


AN EXPERIMENTAL SYSTEM FOR NATURAL COLLOCATED AND REMOTE COLLABORATION

Jian Li,

Department of Civil Engineering, Eastern China Jiaotong University, China.

Jingyu Chen,

Department of Communication, Jiangxi Province, China

ABSTRACT: The phenomenon of convergence between computational and communication technology has created

an absolute breakthrough in the way human interact with each other and how they interact with the technology itself

in particular. In its further development, the convergence has been enhanced to support task-based interaction.

Thorough research on Computer-Supported Cooperative Work (CSCW) and development of groupware are always

complimentary and challenging each other. This abstract proposes an experimental system, a mixture between

electronic peripherals and Human-Computer Interface (HCI) design that allows participants to dynamically interact,

collaborate and communicate with each other, as well as with the system itself. Every user will be projected in real

size at all other participants’ workspace – all of their body gestures, movements, orientation and verbal

communication can be observed by all other participants and being recognized by the system at the same time. This

proposed system will allow users to collaborate on specific works in effective, yet natural workflow. In the system,

awareness in groupware environment is highly maintained by utilizing various technologies that allow critical

awareness information being passed on to all participants. The awareness clues are generated by visual and audio

information. The system aims to provide multi-device collaboration capabilities through the development of an

online collaboration workspace. The focus of this research is to investigate all possible body gestures that can be

used to interact with the collaborative workspace, including the technique used to identify specific body movements

and relate it to the task inside the workspace. On top of that, the research will discuss about the importance for the

system to recognize orientation of the user.

KEYWORDS: Computer-supported cooperative work, design collaboration.

1. INTRODUCTION

There is a huge challenge in developing a groupware system that will further enhance the effectiveness of

productivity of the users (Regenbrecht et al. 2000; Davidson et al. 1996; Schnabel and Kvan 2002). Interactions in

the shared workspace usually limited and unnatural, being limited by inadequate awareness information can be

provided by the groupware system. In fact, awareness is very important knowledge that sustains collaborative works

(1). If collaborative system is not carefully designed – according to Kvan (2000) – participants might think they are

collaborating, but in fact they are actually cooperating or even compromising because of the system cannot provides

users with the ingredients of successful collaborative works (2). The direction of this research is not about to create a

new type of groupware system, rather than to synthesize the combination of various possible technologies and

emerging knowledge that will allow users to interact naturally with the system and with the other users – using set of

gestures and graphical user interface – regardless of location and task.

CONACT SYSTEM will allow users to interact with digital and physical objects in the workspace synchronously,

thus creates an unlimited interactivity between users and those objects. In broader scope, it combines technology in

computational system, communication, display and visualization, multi-modal input, image processing and Human-

Computer Interface (HCI). The synthesis of these peripherals will create innovative interactivity between users and

the virtual collaborative environments on elevated degree. By using CONACT SYSTEM, participants not only able

to see each other in full dimension, but also able to see the entire workspace, every objects in it and everything

happened in the workspace simultaneously. The system will recognize body gesture – the main input and interaction

method in CONACT SYSTEM. By then, participants provided with more awareness clues and natural interactions.

Therefore, it promotes communication that is more effective, enhances the entire workflow and interactivity during

the collaboration process. The focus of this research is to outline all possible body gestures that can be used to

interact with the collaborative workspace, including the technique used to identify specific body movements and

91


relate it to the task inside the workspace. On top of that, the research will discuss about the importance for the

system to recognize orientation of the user.

2. OBJECTIVE, ISSUES, AND SAMPLE CASE According to research conducted by Gutwin and Greenberg (2002), situational awareness of one another is relatively

easy to maintain and the participants can collaborate naturally in spontaneous and effortless. In contrast, awareness

in groupware environment is much harder to maintain compared to face-to-face communication (1). The way that

users operate the system and engaged with other participants are, also important elements that help promote natural

interaction in collaborative environment (3). In CONACT SYSTEM, awareness in groupware environment is highly

maintained by utilizing various technologies that allow critical awareness information being passed on to all

participants. The awareness clues are generated by visual and audio information. Visual information is consists of

the real-size projection of all participants, including their facial expression, body movements and orientation relative

to the workspace. Audio information is consists of all verbal conversation and all sounds originating from any

objects in the workspace. Additionally, intuitive interface is also an important feature in the operation of CONACT

SYSTEM.

CONACT SYSTEM incorporates high-resolution large screen display and vertical display with touch recognition. It

deeply relies on motion tracking sensor and cameras to both capture and recognize the body movement and gestures.

It also combines several microphones to capture participants and environment sound. These devices will capture all

necessary awareness information for all participants. At the central of CONACT SYSTEM, a main control unit

controls the whole system. CONACT SYSTEM can be used as stand-alone collaboration device, or connected with

other computer to extend its functionality. Therefore, the system should be highly compatible with various software

applications, operating systems and hardware platform. CONACT SYSTEM has to be connected to the Internet by

high speed broadband connection in order to supports its data-intensive operational.

This research project explains the importance of utilizing all of those technological devices to deliver adequate

awareness information to the participants and to interact naturally with the workspace through verbal commands and

body gestures. Therefore, participants may be engaged naturally in virtual collaborative environment – effortless and

spontaneously.

2.1 Issues Most of groupware applications cannot handle coordination, cooperation and communication at the same level (3).

Some of them only cover one or two aspects. For example, video conferencing systems are mostly only promotes

communication aspects, while lacking of coordination and cooperation among participants. In different case, online

document editor only assists coordination and cooperation while lacking communication support among

participants.

Nevertheless, the way of users engage with the system and every object in the workspace is also a defining factor of

natural interaction. In face-to-face collaboration, there is no boundary between participants and the environment.

However, in computer-mediated collaborative environment, participants are separated by the system itself and they

cannot interact like in face-to-face collaboration process. In order to create a more sophisticated groupware system,

coordination, cooperation and communication must exist simultaneously at maximum level. This means, participants

can have a good coordination in doing collaborative task, cooperate with others in completing specific task, as well

as communicate with other participants seamlessly. In addition, the system should not restrain participants from

doing collaborative tasks and interacting with every object in the workspace. Therefore, body gesture recognition

will allow users not to deal with complicated operational method of the device, leaving them to collaborate naturally

with other participants. Apparently, most peripherals needed to construct such system are available on the market.

Therefore, it can be assumed that the limitation of building CONACT SYSTEM is rather on arranging and engineer

those peripherals to achieve the qualities mentioned in prior discussion.

2.2 Sample case It is imaginable how hard it is to collaborate remotely on large-scales, graphical-intensive task such as architectural

planning, product design, and 3D animation. All of these tasks require extensive graphical information, as well as

high level of coordination and communication between people involved and cooperation towards the tasks. In such

92


condition, complete view of the workspace and other participants, as well as naturalistic interaction with the system

will benefit all participants in the work process. The system offers a close-to-reality experience in a virtual platform.

In other way, the system might be quite redundant in supporting less complex collaborative tasks and

communication, such as application sharing and video conferencing. However, the fundamental functions possessed

by the system will benefits participants, especially regarding the gestural interaction with the system itself – which

arguably the future of human-computer interaction method.

3. SPECIFICATION

3.1 Technology involved In any CONACT SYSTEM set, there is vertical large-screen display, horizontal display with touch-recognition,

video camera to capture the image of participants, motion sensor and image tracker for gesture recognition,

microphones, speaker system and also controller unit – which is the ‘brain’ of CONACT SYSTEM. Figure 1 depicts

the entire system architecture and setup.

Figure 1. The system architecture

93


3.1.1 Vertical large screen display

The vertical large-screen display is where the images of other participants are displayed. It can utilize either large

panel (> 70 inch) high-resolution LCD display, or LCD projectors. All display devices must have high contrast ratio

and brightness to withstand operational under bright light.

3.1.2 Horizontal (table top) display with touch recognition

Tabletop display is the main place for interaction using CONACT SYSTEM. It also supports touch recognition.

Display resolution is also important – the higher the resolution, the bigger the workspace area that can be accessible

by users. It is also preferable to use high contrast-ratio and brightness display. In addition, operational temperature

of the device must not disturb the users, especially for long period of use.

3.1.3 Camera and motion tracker

The camera is used to capture live video of the participants and the workspace area. It supports high frame-rate

recording to reduce ghosting effects. Other camera – mounted above the tabletop display – can be used as ‘document

camera’. It can take a picture of objects placed on the tabletop display. Therefore, users are not only able to interact

with digital objects, but also with physical objects. Motion tracker is used to read body movement and orientation,

especially hand gestures. Motion tracker can be coupled with special gloves that can read fingers movements of each

user, in order to enhance the accuracy of the recognition.

3.1.4 Audio system

In CONACT SYSTEM, the speaker system must be able to produce sound relative to the location of the object

appeared on the vertical screen, as well as on the tabletop display. Several microphones are used to capture

participants’ voice and sounds in the workspace area. There is ambient noise reduction function to help reducing all

unnecessary noises (such as air condition noise, noises from outside of the room, reverbs, echoes, etc.) and therefore,

allows users to enjoy a clear voice communication.

3.1.5 Control unit The control unit is a standard computer with custom-made software platform that control the whole CONACT

SYSTEM and do all processing necessary for the operational of the system, including the image processing. It

supports multi-display output, multi-channel sound input and output. It also has both wired and wireless

communication interface to connect with other devices, local networks and the Internet. All components of

CONACT SYSTEM – including display units, motion trackers, cameras and audio devices –are connected to this

control unit.

3.1.6 Ambient light sensor

Light sensor is added to the system, so it will automatically alter the display brightness to adapt with the light

intensity in surrounding area. Thus, users can always have the right display brightness and comfortable view at the

displays.

3.2 Gesture recognition The interaction of users with CONACT SYSTEM, is primarily depends on gestural recognition of hands, including

finger touches as well as head and eyes orientation. The recognized gestures are classified into five categories:

native function, basic control, media control, object manipulation and object handling. Each category can

complement each other, but not necessarily related. Usages of gestures are not fixated, but allow combination in

different situations. Table 1 listed all the enabled gestures.

In order to simplify the user interface on each end, only the finger orientation can be seen by other participants.

Other gestural exposures and all user interface that triggered by that particular gesture are not necessarily observable

to other participants. However, other participants should be able to see the actions and reactions caused by certain

gestures. For example, finger orientation will always be visible for all participants, but if a participant rotating and

scaling a picture, the effect of rotation and scaling on the picture can be seen by other participants.

94


Table 1. List of Gestures

3.3 Design plan CONACT SYSTEM is not intended for portable use and must be installed in a specific room that can accommodate

space required by the minimal projection distance between the projectors and the screen, as well as the appropriate

distance between the screen and the interactive tabletop. For detailed illustration, see Figure 1.

95


3.4 Operational and usage As has been previously discussed, user interacts with the CONACT SYSTEM by primarily using gestures and

touchable graphical user interface. The whole system is controlled by main control unit and connected to high-speed

internet connection for a seamless audio and video communication. A moveable touch screen is available for the

control panel interface for the whole system. At the front, a large-scale display is intended to projects the realistic

size of the other participants and there are several cameras in front of every users for the video communication.

Special software in the main control unit will simulate the gazing direction of a user to other particular user, creating

a more natural communication that closely resembles real-life size objects.

CONACT SYSTEM can be used as a standalone device, as an extension for other computer device, as well as

connected to the local computer network. In addition, other computers and compatible mobile devices can be

connected by both wired and wireless connectivity. The software platform (driver) loaded on those devices will

allow file exchange between the external device and CONACT SYSTEM. CONACT SYSTEM is suitable for

collaborative design, architecture and engineering works, and possibly valuable for remote education device, or just

as a state-of-the-art video conferencing device.

3.5 Evaluation Parameter As CONACT SYSTEM is only a concept product, it is rather hard to evaluate the usability and effectiveness of such

system. However, the most likely approach of this evaluation process is by using real test case for collaboration in

two remote places, involving at least two participants on each place. To achieve this evaluation procedure, the whole

system must be physically built as working prototype form. While doing the evaluation, the participants should be

encouraged to give feedbacks on the easiness of operating the system and how much effort they have to understand

the operational method (hand gesture, graphical user interface, etc.) of CONACT SYSTEM. The hand gesture must

be assessed as well to be easily understandable, even by new users and the graphical user interface must be assessed

for its ergonomic and clarity.

In addition, as the proposed CONACT SYSTEM framework is huge and complicated, there must be many prior tests

on software and hardware reliability to ensure minimal problems in real-life usage. The bottom line of the evaluation

process is how the system giving benefits to the client, compared with the high price tag. If the clients find that the

system could improve their working efficiency on data-massive, graphical intensive collaborative session, then the

system can be assumed as successful.

4. REFERENCES

Davidson, J.N. and Campbell, D.A. (1996). “Collaborative Design in Virtual Space - GreenSpace II: A Shared

Environment for Architectural Design Review, Design Computation: Collaboration, Reasoning, Pedagogy.”

Proceedings of ACADIA Conference, October 31 - November 2, pp. 165-179.

Gutwin, C. and Greenberg, S. (2002). “A Descriptive Framework of Workspace Awareness for Real-Time

Groupware”. Computer Supported Cooperative Work, Special Issue on Awareness in CSCW, Kluwer Academic

Press, 11 (3), pp. 411-446.

Kvan, T. (2000). “Collaborative Design: What is It?”, Automation in Construction, Vol. 9, pp. 409-415.

Regenbrecht, H., Kruijff, E., Donath, D., Seichter, H., and Beetz, J. (2000). “VRAM - a Virtual Reality Aided

Modeler”. Proceedings of eCAADe2000, Weimar/Germany.

Schnabel, M.A. and Kvan, T. (2002). “Design, Communication & Collaboration in Immersive Virtual

Environments”, International Journal of Design Computing, Volume 4; April 2002; ISSN 1329-7147.

96


URBAN WIKI AND VR APPLICATIONS

Wael Abdelhameed, Ph.D.,

University of Bahrain, College of Engineering, Bahrain;

South Valley University, Faculty of Fine Arts at Luxor, Egypt;

[email protected]

Yoshihiro Kobayashi, Ph.D.,

Arizona State University, School of Architecture and Landscape Architecture;

[email protected]

ABSTRACT: The research paper involves the implementation of Urban Wiki, an online urban design system

employing Wiki concept, allowing the use of an interactive immersive virtual reality system for visualizations with

dynamic agents such as human and vehicular traffic. The VR system is a platform developed by a software company.

The term Urban Wiki is created by the researchers. Urban Wiki aims to creating a networking system of urban

designs, enabling the collaborative work between users around the world through a VR platform. The presented

system framework is created and tested by the researchers from two different locations in the world. The purpose of

the research is to study how the users can share effectively designing a large scale urban project, and how VR

platform helps in building up the VR urban models to facilitate visualizations and designing. An urban project of a

village scale, which was conducted by one of the researchers is used to demonstrate the potentials of Urban Wiki,

presenting its functions and highlighting the possible uses in the urban area. Moreover, using the created models in

the VR platform that enables visualizations with dynamic agents opens various urban paths of designing, decision-

making, sharing, and communication with the stakeholders, decision makers, and planners. Techniques employed in

the design of Urban Wiki can be potentially used to build scalable, easily navigable and extensible models of large-

scale entities. Combining the application of the two systems, Urban Wiki and VR platform, will be designed as an

intuitive simulation tool, helpful in identifying novel approaches for control and visualization in such applications

as urban design, urban plan and Land Use –Physical- Plan.

KEYWORDS: Urban Wiki, VR Applications, VR Platform, Collaborative Design.

1. INTRODUCTION

Collaborative design concept enables users to modify the content of a file from different places around the world.

The available modifications at the beginning were exclusive to the text format, and then other formats such as image

were adapted. Yet, the collaboration in the designing process itself especially on the 3D level is not effectively

introduced. Editing or modifying a 3D design file through a computer system that displays the content of this 3D

design file has not been investigated.

Although there is no enough literature in the area of 3D collaborative designing, some researches investigated

similar areas. For example, Yamashita et al. (2006) developed a collaborative design environment which considers

Information and Communication Technology and architectural space, through supporting synchronous design

collaboration in a face-to-face meeting at a local site and also in a continuously connected project-room at

distributing sites (Yamashita et al., 2006). Lan and Chiu (2006) demonstrated a Web 3D-GIS approach to develop

the urban information system. Lan and Chiu proposed that a digital city should be able to not only visualize a large-

scale 3D city model but also integrate useful urban information for potential users’ retrieval in a web environment

(Lan and Chiu, 2006). Matsumoto, Kiriki, Naka, and Yamaguchi (2006) proposed the collaborative design education

program on the web, and developed the special Design Pinup Board system for running it. The introduced program

focuses on very limited environment; distributed collaboration beginners, asynchronous, first meeting, and plural

teams (Matsumoto, et. al., 2006). Lee (2001) maintained the possibility to create a 3D modeling tool based on the

recognition of labels in freehand sketches, and introduced a symbol-based 3D modeling tool (the SpaceMaker) that

allows designers to make freehand floor-plan drawings to explore the initial concept of spatial layout and allows

users to apply labels to identify different types of space (Lee, 2001).

97


Offering the opportunity to modify a 3D design through a networking system at different locations around the world

has been effectively introduced only on the commercial level, such as Secondlife and CityEngine. Second Life is a

3D virtual world. Its residents are allowed to build, own and retain the rights to their digital creations. They,

therefore, can buy, sell and trade with other residents. The Marketplace currently supports millions of US dollars in

monthly transactions. Another example related to the same approach is Massively Multiplayer Online Role-Playing

Game (MMORPG) Ultima Online that was created in 1997. CityEngine is 3D modelling software for urban

environments. It enables its users to build their own neighbourhoods, urban areas and cities with certain types of

buildings and houses.

Linking the concept of collaborative design on the 3D level to the famous Wiki concept is one of the concerns of

this research. Urban Wiki is an online urban design system employing the Wiki concept. The research concern,

moreover, is to investigate the use of an interactive immersive virtual reality system during employing the Urban

Wiki.

2. RESEARCH OBJECTIVES

Urban Wiki implements the requirements and objectives that can be summarized in the following:

• Sharing an urban plan through/in a file of max scripting;

• Using a 3D modeling system. Its transformations and changes of form assembling are through only

three simple buttons (create, edit and delete);

• Applying functions of searching (zoom in and out) and modifying (create, edit and delete) through

networking; and

• Linking the urban file to a VR platform.

3. WIKI AND URBAN DESIGN

A Wiki is a website that allows visitors to add, remove, edit and change its content, typically without the need for

subscription. A wiki is an effective tool for mass collaborative authoring through this easiness of interaction among

its visitors. Urban Wiki of the same Wiki potentials is investigated by the researchers in another research paper.

Moreover, Wiki concept on the design level, Design Wiki, was previously introduced by the researchers*******.

Design Wiki has a networking 2D/3D visual design map, DesignMap, through which Design Wiki visitors can edit

the existed designs and then save the modified designs in series based on their topological properties.

The main objective of this research is to share effectively the designing process of a large scale urban project, and to

allow visualizing the urban file through a certain VR platform. Therefore, Urban Wiki investigated by this research

is focused on activities of a group of interest during conducting an urban planning project.

3.1 Urban Wiki

The public Web portals that appeared in the mid-1990s, such as Yahoo, Msn, AltaVista, and Excite, have portlets

that provide self-contained boxes for different features like e-mail, news, weather and search. By the late 1990s,

software vendors began to produce pre-packaged enterprise portals, which would be toolkits for enterprises to

quickly develop and deploy their own customized enterprise portal. There are many enterprise portal vendors such

as, Apache Software Foundation (its product name is Jetspeed 2.1), IBM (its product name is WebSphere Portal

Server 6.0.1), and Microsoft Office (its product name is Sharepoint Server 2007).

Urban Wiki is programmed as a main Portlet. Urban Wiki portlet, which is programmed in Java, is a reusable

interface for online applications, and it is running with JetSpeed2 framework provided by Jakarta Project. The

portlet can be run through all kinds of Portal applications. It was tested using a free Enterprise Internet Portal

Framework, JetSpeed2 (by Apache Portal Project site, http://httpd.apache.org/).

3.1.1 Methodology

The user has to prepare two files; an initial space layout file and a file of space property list. The file of Space

Property List has the adjacency list of a design, for example in Figure 1, the adjacency list for 43 is: 8, 22, 40, 39,

30, and by changing the three outlined cells to a new space, e.g. 45, the list will be: 8, 45, 8, 22, 40, 39, 30. The

98


Space Layout file should have at least one design with an array of 400 (20x20) integer numbers such as (0 0 0 0 1 1

1 …. 0 0 0).

FIG. 1: A displayed 2D design with the three buttons of modifying by adding the three outlined cells

Each integer number represents a space property such as 0=Street, 1=Sidewalk, and 2=House, which are defined the

file of space property list, Figure 2. By changing the properties of cells in the portlet’s grid, new spaces are created.

A space is defined as a set of cells with the same property such as house, front yard, driving way, and garage.

FIG. 2: A design displayed in 2D and 3D with a space list

99


After creating the design or the urban plan, it can be added to the main urban file of max scripting. Visitors, a group

of interest, who have the access to the main urban project file can add, edit or delete designs of this urban project.

Figure 3 presents a 2D urban plan project that was used to test the application.

FIG. 3: 2D maps, land use and master plan, of the urban plan project, Arabah Sharqeyah Village, Sohag

Governorate, Egypt.

3.1.2 Functions

The functions offered by Urban Wiki were tested by the researchers through making modifications at two different

locations to the urban file. The interactive modifications through the VR allow to visualize the impact of these

modifications and to link them to other factors. The influential factors assessment and their impact in an interactive

VR environment help not only urban designers but also the associated partners and stakeholders.

3.1.3 Abstract outputs

Urban Wiki applies only square grids which would generate straight shapes and forms. Although diagonal lines and

polygonal shapes can be adapted and implemented by Urban Wiki, the research paper employs an abstract style for

the created designs possibly added to the urban models. Whereas the focus on the large scale projects of urban

designing and urban planning is directed to the surrounding environment rather than buildings and their details.

100


3.2 The Urban Project

On large scale urban projects, many authorities and individuals are involved, for example municipalities, district

councils, stakeholders, decision makers, and planners. A major part of the designing process itself is the effective

communications between the different parties. Facilitating the processes of sharing, communication, decision

making, and visualization is the main advantage of employing the Wiki concept in urban design projects.

The urban project presented by this research is of Arabah Sharqeyah, a Village at Sohag Governorate in the middle

of Egypt. During this urban project, there were several meetings with the inhabitants of the village and the members

of village council in order to share in the decision making processes whenever it is approved by the planning

authorities and the Ministry of Housing and Planning.

The urban model was created by the 3ds max file and displayed in Urban Wiki system, Figure 4. The final output

that has modifications made by the Urban Wiki is imported to the VR program platform in max scripting file format.

The link between Urban Wiki and a VR platform eases many activities and tasks of the urban designing. Of the

previous experience resulted from conducting this urban project, the VR models with the possibility of interaction

offer a more effective way than of provided by the static models.

FIG. 4: The user interface of the Urban Wiki System.

101


Based on the Space file used in the Urban Wiki, each design can present certain data. This flexibility is important to

adapt different styles of urban fabric of different urban projects. Different models would be built up to visualize and

evaluate the alternative solutions and various influential factors of the urban environment. These changes and

modifications can be visualized through the Urban Wiki system at different locations at the same time through

networking.

4. VR PLATFORM

The final step is to import the main urban project file to a VR platform. The platform used is UC Win/Road, a

software program developed by the Forum8 Company. Its version 3.04 has the function of importing different file

formats such as shp, max script, 3ds max and dwg. Urban Wiki employs the option of max script format. Also, the

latest version, VR studio, has the function to visualize the same VR model through networking at computers of

different locations. Figure 5 shows a screenshot of the VR model of an urban project, displayed in UC Win/Road.

FIG. 5: A screenshot of the VR program showing the village model.

The VR platform, UC Win/Road 3.04, enables visualizations of traffic simulation with dynamic agents such as cars

and pedestrians. This interactive immersive virtual reality system opens various urban of designing, decision-

making, sharing, and communication with the stakeholders, decision makers, and planners. These applications of

Virtual Reality platform are the concern of this research. Various urban project problems can be solved while

different scenarios can be visualized for evaluation and comparisons.

On another research contribution, techniques employed in the design of Urban Wiki can be potentially used to build

scalable, easily navigable and extensible models of large-scale entities. On a village scale or district level, navigation

102


through the whole project in terms of adding or modifying certain urban areas is highly important. Providing this

function at different locations and visualizing the outputs within these locations at the same time, through Urban

Wiki system, are a major part in the design process of urban projects.

5. SUMMARY AND DISCUSSION

The research concludes to a unique urban system, Urban Wiki, where its objectives, functions and methodology

were illustrated. The contributions and applications of Urban Wiki, through networking and its possible link to a VR

platform can be employed effectively in urban planning as tools of designing, decision-making, sharing, and

communication. VR models are improving the practice of urban environmental planning and design. The visual

display capabilities enable the explanation of the development plans, alternatives or various scenarios to both the

urban project team and the public.

Although the research paper did not concentrate on the urban planning project itself, some modifications were made

in the urban design at different locations –countries- through networking in terms of testing and simulating the same

conditions during conducting the project.

There are areas and factors which can be considered crucial urban planning issues that present focuses of future

research papers to be investigated on the urban level.

6. ACKNOWLEDGEMENTS

This research was funded through the research project 4/2008 by the University of Bahrain. We would like to

express our appreciation to the University of Bahrain. Also, our thanks is to Forum8 company for the help they

offered.

7. REFERENCES

Kobayashi Y. and Abdelhameed W. (2008) “Implementations and Applications of DesignMap: Case Studies to

Manage an Online Database of Visual Designs” International Journal of Architectural Computing, vol. 6 -

no. 3, 243-258.

Lan U-H. and Chiu M-L. (2006) “A Web 3D-GIS Approach to Develop the Urban Information System of Virtual

Anping.” Proceedings of the 11th Conference on CAADRIA, Kumamoto, Japan, 479-486.

Matsumoto Y., Kiriki M., Naka R. and Shigeyuki Y. (2006) “Supporting Process Guidance for Collaborative Design

Learning on the Web: Development of Plan-Do-See cycle based Design Pinup Board.” Proceedings of the

11th Conference on CAADRIA, Kumamoto, Japan, pp. 71-80.

Yamashita, S., Yoshitaka M., Yuji M., Ryusuke N., and Shigeyuki Y. (2006) “Enhanced and Continuously

Connected Environment for Collaborative Design” Proceedings the 24th Conference on eCAADe,

Communicating Space(s), Volos (Greece) 2006, pp. 478-485.

Lee M.-C. (2001) “SpaceMaker: A Symbol-based Three-dimensional Computer Modeling Tool for Early Schematic

Development of the Architectural Design.” M.Sc. Thesis, Design Machine Group, University of Washington,

Washington, USA.

http:www.secondlife.com/ [15-7-2009].

http:www.cityengine.com/ [15-7-2009].

http:www.forum8.co.jp/english/english0.htm [15-7-2009].

103

AUTOMATION AND INTERACTION

Toward Affective Handsfree Human-machine Interface Approach in

Virtual Environments-based Equipment Operation Training--------------------------107

Iman Mohammad Rezazadeh, Xiangyu Wang, Rui Wang and Mohammad Firoozabadi

Construction Dashboard: An Exploratory Information Visualization Tool

for Multi-system Construction----------------------------------------------------------------117

Cheng-Han Kuo, Meng-Han Tsai, Shih-Chung Kang and Shang-Hsien Hsieh

Computer Gaming Technology and Porosity-----------------------------------------------127

Russell Lowe and Richard Goodwin

Virtual Reality User Interfaces for the Effective Exploration and

Presentation of Archaeological Sites----------------------------------------------------------139

Daniel Keymer, Burkhard Wünsche and Robert Amor

Interactive Construction Documentation----------------------------------------------------149

Antony Pelosi

Case Studies on the Generation of Virtual Environments of

Real World Facilities-----------------------------------------------------------------------------155

Michele Fumarola and Ronald Poelman

Evaluation of 3D City Models Using Automatic Placed Urban Agents----------------165

Gideon Aschwanden, Simon Haegler, Jan Halatsch, Rafaël Jeker, Gerhard Schmitt

and Luc van Gool

Integration of As-built and As-designed Models for

3D Positioning Control and 4D Visualization during Construction---------------------177

Xiong Liang, Ming Lu and Jian-Ping Zhang

Augmenting Site Photos with 3D As-built Tunnel Models for

Construction Progress Visualization----------------------------------------------------------187

Ming Fung Siu and Ming Lu

Automatic Generation of Time Location Plan in Road Construction Projects------197

Raj Kapur and Nashwan Dawood

Development of 3D-Simulation Based Genetic Algorithms to

Solve Combinatorial Crew Allocation Problems-------------------------------------------207

Ammar Al-Bazi, Nashwan Dawood and John Dean

Integration of Urban Development and 5D Planning--------------------------------------217

Nashwan Dawood, Claudio Benghi, Thea Lorentzen and Yoann Pencreach


TOWARD AFFECTIVE HANDSFREE HUMAN-MACHINE INTERFACE

APPROACH IN VIRTUAL ENVIRONMENTS-BASED EUQIPMENT

OPERATION TRAINING

Iman Mohammad Rezazadeh, Ph.D Candidate

School of Biomedical Eng., Science and Research Branch of Islamic Azad University, Tehran, Iran

[email protected], [email protected]

Xiangyu Wang, Dr.

Design Lab, Faculty of Design, Planning and Architecture, University of Sydney, Sydney, Australia

[email protected]

Rui Wang, Master of Philosophy Student

Design Lab, Faculty of Design, Planning and Architecture, University of Sydney, Sydney, Australia

[email protected]

Mohammad Firoozabadi, Prof.

School of Biomedical Eng., Science and Research Branch of Islamic Azad University, Tehran, Iran

Medical Physics Dept. Tarbiat Modares University, Tehran, Iran

[email protected]

ABSTRACT: Using Virtual Reality (VR) technology for training is becoming more and more interesting to many

applications ranging from medical to industrial purposes. In the construction arena, some progresses have been

achieved by researchers to design and implement environments for task training using VR technology and its

derivatives such as Augmented and Mixed Reality. However, there are still many shortcomings in this area, which

should be considered. Usefulness and usability of the virtual training environments are two of the most important

factors when designing and implementing them. The usefulness factor can be achieved by designing the virtual training

environments in a way that it can support training phase or could be a substitute for early stage of training in the real

environment, but eliminate the real environment training drawbacks such as high cost, high risk and high difficulty of

repetitive practices. On the other hand, for usability factor, we should suppress the mental and cognitive pressure and

stress over the user whilst he or she is being trained. In the work presented in this paper, we designed a virtual lift

(crane) which can be controlled using commands extracted from facial gestures and is capable of lifting up

load/materials at virtual construction sites. Then we utilized Electroencephalogram (EEG) signals collected from the

users during training to extract their affective measures, which mirror trainees’ level of comfort. This measurement can

also be used for further tasks such as re-designing the environment and also can be used to stop training until the

trainee reaches the relax state to continue the training phase.

KEYWORDS: Virtual Reality, Construction training, Affective measures, Facial bioelectric signals

1. INTRODUCTION

Construction equipment operators usually operate one or several various types of construction equipment. Virtual

technologies afford new opportunities for effectively training novices with lower cost and fewer hazards. Seidel and

Seidel and Chatelier (1997) have even suggested, for example, that the use of virtual environments (VEs) may be

“training’s future”. Virtual environments can be especially valuable where training in real-world situations would be

impractical because a real field scenario may be unduly expensive, logistically difficult, dangerous, or too difficult to

control. This approach is envisaged to facilitate progress along what is a steep learning curve and enable effective

rehearsal of future operations in actual construction sites. Construction training research has begun to explore Virtual

Reality as training vehicles. Virtual reality training systems have already provided added benefits to many training

packages. Virtual environments (VEs) embody many of the characteristics of an ideal training medium (Psotka, 1995;

Schroeder, 1995); VEs have already been developed for training of drivers (Mahoney, 1997), firefighters (Bliss,

Tidwell, & Guest, 1997), pilots (Lintern, Roscoe, Koonce, & Segal, 1990), console operators (Regian, Shebilske, &

Monk, 1992), naval officers in ship manoeuvres (Magee, 1997), soldiers in battlefield simulations (Goldberg & Knerr,

1997; Mastaglio & Callahan, 1995), and ground control team for familiarity with the operability of the Hubble Space

Telescope (Loftin, et al., 1997).

107


Apart from the noted existing VR research in training, recently, it becomes an important research issue in the robotics

field to recognize emotional states such as joy, teasing, fear, sadness, disgust, anger, surprise, and neutral from human

bioelectric signals. A number of studies have been done in this area, ranging from psycho-physiological measures such

as heart rate, EDA pupillometrics or fEMG to speech or video analysis. From different kinds of survey results such as

questionnaires and interviews, it is clear that the success rates are 50-60% in emotional speech recognition and 80-90%

in facial expressions. As we know, physiological indexes are useful to evaluate emotions since they can be measured

physically and objectively and can be easily applied to engineering approaches. One of these approaches which have

been gaining many attentions is affective computing. Affective computing basically means computing that relates to,

arises from, or deliberately influences emotions and it focuses on creating personal computing systems having ability to

sense, recognize and understand human emotions, together with the skills to respond in an intelligent, sensitive and

respectful fanner toward the user and his emotions (Fernandez, 1997; Healey & Picard, 2000; Nasoz, Lisetti, Alvarez, &

Finkelstein, 2003; Picard, 1997).

Ang et al. (2004) stated that facial muscle movements and forehead electromyogram (fEMG) can be corresponding to

certain facial expressions and are the most important visual representation of a person's physical emotional states.

Mahlke and Minge (2006) used emotional states which were extracted from fEMG to discriminate between usable and

usable computerized contexts and they concluded that the frowning activity is significantly higher in the unusable

system condition than in the usable one by pacing two pairs of electrodes on zygomaticus major and corrugators

supercili to detect positive and negative emotional states, respectively. Neimenlehto et al. (2006) studied the effects of

affective interventions using fEMG in a Human-Computer Interaction (HCI) and concluded that the frowning activity

attenuated significantly after the positive interventions than the conditions with no intervention. Kim et al. (2006) used

fEMG Linear Prediction Coefficients (LPCs) by placing a pair of electrode on temporalis muscle and control an electro-

powered wheelchair (EPW) using clenching left, right and both molar teeth and eye blink and classified them using a

Hidden Markov Model (HMM) and achieved 96.5% and 97.1% decimation rate for handicapped and healthy groups,

respectively. Nasoz et al. (2003) have conducted a study to model user emotional state. Three physiological

measurements were used and data was collected from 31 participants (male and female) from student population. This

study used normalized signals instead of statistical features and employed two separate classification method k-nearest-

neighbor (KNN) and Discriminant Function Analysis (DFA). The best results were achieved using DFA to classify 5

emotions with 90% accuracy for fear, 87.5% for sadness, 78.58% for anger, 56.25% for surprise and 50% for

frustration. Healey and Picard (2000) have used 4 physiological signals to detect the intensity of stress in automobile

drivers. Sequential Forward Floating Search (SFFS) was used to recognize patterns of stress whereby the intensity of

stress was recognized with 88.6% accuracy. Fernandez (1997) used two physiological signals to detect frustration in

computer users.

This paper is dedicated essentially to the creation of compelling virtual environments within which human participants

are led to feel somehow present, for purposes of training. More specifically, a novel facial multi-channel bio-electric

signals processing approach was developed to extract affective measures while performing some pre-requested and

sudden tasks. The results of this research could enable better development, implementation, and assessment of virtual

environments for equipment operators. Given the size of the construction industry and other related industries (e.g.,

manufacturing), the results of this research are expected to directly impact workforce and economy. Section 2 presented

the novel facial multi-channel bio-electric signals processing approach to extract affective measures. Section 3

discussed the results from the preliminary study based on the approach.

2. METHOD

The general block diagram of our method to design a bioelectric interface is depicted in Figure 1.

108


Figure 1 – General Block diagram of the proposed system

2.1 Site selection and electrodes placement

Before the placement of any electrode, the selected area should be cleansed from dust, sweat and fat layer to reduce the

effect of motion artefacts. As illustrated in Figure 2, three pairs of rounded pre-gelled Ag/AgCl electrodes were placed

on the volunteer's facial muscles in a differentiation configuration to harness the highest amplitude signals (Firoozabadi

et al., 2008):

• One pair on his frontalis muscle: above the eyebrows with 2cm inter-electrodes distance. (Channel 2)

• Two pairs placed of left and right temporalis muscles. (Channel 1 and 3)

• One ground electrode on the boney part of the left wrist.

2.2 Data acquisition system

The Biopac system (MP100 model and ack100w software version) (Biopac, 2009) was used to acquire bioelectric-

signals. It can collect bioelectric-signals accurately with the selected sampling frequency and store them in its own or

PC memory (1.73 GHz, 2G RAM). The sampling frequency and amplifier gain were selected at 1000 Hz and 5000,

respectively. The low cut-off frequency of the filter was chosen 0.1 Hz to avoid motion artefacts. In addition, a narrow

band-stop filter (48Hz-52Hz) was used to eliminate line noise.

109


Figure 2- Illustration of the electrodes configuration over frontalis and temporalis facial muscles (Firoozabadi et al.,

2008)

2.3 Data collection protocol

A pilot study was conducted to test the effectiveness and usability of this affective computing-based virtual training

system. Three healthy volunteers have been chosen for this study (male and aged 22, 25, and 29 years old respectively).

Each volunteer rested for 5 minutes prior to recording session. Then he was asked to moderately perform the facial

gestures according to Table 1, 10 times (trails); the data from the 3 channels (3 pairs of facial electrodes) was then

recorded for a period of 2 seconds and started 1 second after gesture generation. There was a 10-second interval

between each trail in order to eliminate the effect of fatigue.

Table 1. Gesture Name and Related movement.

Related Command Gesture Name Gesture Index

No.

Move Forward Smiling 1

Move Right Pulling up right lip corner 2

Move Left Pulling up left lip corner 3

Move Backward Opening mouth (like to say 'a' in 'apple') 4

Lift/Release the load Clenching Molar teeth 5

2.4 Data pre-processing and manipulatory feature extraction

The acquired data from the three channels was passed through a band-passed butterworth filter ranges 30-450Hz which

covered the most significant spectrum related to facial electrical activity. Then the data was divided into non-overlapped

256msec slots and for each slot the root mean square (RMS) value was calculated as a manipulator feature (Ri).

(Formula 1)

€

Ri= RMS(EXG

i) =

EXGi

2dt

0

T

∫T

(1)

110


Those manipulator features whose values were greater than the threshold, were considered as active features (Ti)

(Formula 2).

€

Ti = Ri ≥ 3Mean RMS EXGQuiescent( )( ) + 3std RMS EXGQuiescent( )( ){ } (2)

Finally, the normalized manipulator features were achieved using Formula 3:

€

Si=Ti−Mean RMS EXG

Quiescent( )( )(T

i−[ ]RMS EXG

Quiescent)( )

i=1

K

∑ (3)

2.5 Classification

Extracted features need to be classified into distinctive classes for the recognition of the desired gesture. In addition to

inherit variation of bioelectric-signal over time, there are external factors, such as changes in electrodes position,

fatigue, and sweat which may cause changes in a signal pattern over time. A classifier should be able to cope with such

varying patterns optimally, as well as prevent over fitting. Classification should be adequately fast to meet real-time

processing constraints. A suitable classifier has to be efficient in classifying novel patterns; online training can maintain

the stably of classification performance over a long-term operation (Oskoei and Hu, 2007).

The idea of fuzzy clustering is to divide the data space into fuzzy clusters, each representing one specific part of the

system behaviour. Fuzzy c-means is one the fuzzy clustering methods which is a supervised algorithm, because it is

necessary to tell it how many clusters c to look for. If the number of centres is not known before, it is necessary to apply

an unsupervised algorithm. Subtractive fuzzy-means clustering (SFCM) is based on the measurement of the density of

data points in the feature space. The idea is to find regions in the feature space with high density of data points. The

point with the highest number of neighbours is selected as centre for a cluster. The data points within a pre-specified,

fuzzy radius are then removed (subtracted), and the algorithm looks for a new point with the highest number of

neighbours. Subtractive clustering uses data points as the candidates for cluster centers, instead of grid points as in

mountain clustering. This means that the computation is now proportional to the problem size instead of the problem

dimension. Based on the above description, the k-folds algorithm was applied to the training set; where k equals to 10.

The k-1 folds were used to train the classifier and applied to SFCM to derive fuzzy inference system and the rest 1 fold

used to validate it (Moertini et al., 2002; Priyona et al., 2003).

2.6 Affective feature extraction

The Wavelet Packet Entropy (WPE) appears as a measure of the degree of order/disorder of the signal, so it can provide

useful information about the underlying dynamical process associated with the signal.

Assume that wavelet coefficient are given by (4) where S and are the main signal and the corresponding kernel at

shift k and scale j; j=-1,…-N; the energy at each resolution level will be the energy of the detail signal and the energy at

each sampled time k will be (5). In consequence, the total energy can be obtained by (6).

€

C j (k) = S,Ψj ,k (4)

€

E↓ j = ↓∑ k ≡ |C↓k(k)|↑2[ ] (5)

€

E(k)[ ]↓

= ( j = −N)↑↓

∑ (−1) ≡ |C↓ j(k)|↑2[ ] (6)

The relative wavelet packet energy for the resolution j is defined as (7.1 and 7.2):

€

Etot S2

= E j

j<0

∑ (7.1)

€

Pj =E j

Etot

(7.2)

111


Now, the Shannon entropy gives a useful criterion for analyzing and comparing probability distribution, providing a

measure of the information of any distribution given by (8).

€

SWT = SWT (p) = − p j

j<0

∑ • ln(p j ) (8)

The acquired data from Channel 2 was passed through a 13-21 Hz band-passed filter to gain beta band of brain

electrical activity, which is related to mental activities. Then the filtered data was divided into non-overlapped

1000msec time slots and The Wavelet Packet Entropy (WPE) method was applied the slots to obtain relative entropy

for each pre-defined task known as affective features.

The main reason to choose Channel 2 for acquiring the affective measures is it can capture forehead bioelectric signals

and according to manipulation commands, there was no source of manipulation commands in the forehead.

2.7 Virtual crane and its control Two different virtual environments have been created. The first one (VE1) was an in-house virtual crane built by

MAYA and it could lift, move and place a virtual load according to manipulation commands as seen in Figure 3.

Figure 3: VE1: virtual crane could be used to place the load on the other side of the wall

Figure 4: VE2: virtual portal crane could be used to place the load in different locations

The second virtual environment (VE2) was also a portal crane from HUMUSOFT s.r.o. and the MathWorks, Inc.(

Figure 4). It could lift the virtual load and be controlled via maipulatory commands from facial gestures to make the

desired trajectory of movement.

3. EXPERIMENTAL RESULTS AND ANALYSIS Each of the three volunteers were asked to perform data collection protocol. Table 2, shows the achieved result to

discriminate the mentioned facial gestures. It is clear that the classifier has good power to discriminate between facial

gestures. Also, the generating of these gestures are natural and that is why this gesture combination could be used as a

good interface for human machine applications.

112


Table 2. Average discrimination ratio for mentioned facial gestures for 3 users Discrimination

Ratio %

Gesture Name Gesture Index No.

90 Smiling 1

85 Pulling up right lip corner 2

96 Pulling up left lip corner 3

100 Opening mouth ( like to say 'a' in 'apple') 4

92 Clenching Molar teeth 5

Then, each human subject was asked to control VE1 and VE2 according to following protocols:

- VE1: he should lift the load and pass it through the wall release it in the other side of the wall

- VE2: he should lift the load, pass it to the opposite corner of the ground and release it.

Both VE1 and VE2 were set to be performed in three different levels of difficulties:

- Level 1(Easy): normal crane movement speed

- Level 2 (Difficult): Fast crane movement speed

- Level3 (Difficult): Slow crane movement speed

Figure 5 shows that there is difference in entropy value when performing Level 1 (blue), level 2 (red) and level 3

(green). That is because the mental and cognitive stress were more higher when performing more difficult task and

Channel 2 could be considered as a good affective channel which can mirror user internal feeling about the protocol.

Also, the average time to complete VE1 and VE2 considering all levels of difficulties are 12.21 sec and 17.68 sec.

Figure 5- Average entropy for Beta band levels of different levels of difficulties for VE1 and VE2. (Blue: Level 1, Red:

Level 2 and Green : Level 3) for case 1 and case 2.

4. CONCLUSION AND FUTURE WORKS

In the work presented in this paper, we developed our previous work on facial multi-channel bio-electric signals

processing approach (Firoozabadi, Oskoei , & Hu, 2008) to extract affective measures while performing some pre-

requested and sudden tasks. In order to test and justify this approach, a virtual environment was created in the presented

study where a virtual carne has been built which can be controlled and manipulated using facial bioelectric signals. A

pilot study was implemented by inviting three human subjects and data was collected. These signals were captured

when facial gestures were generated by users. By using Channel 1 and Channel 3 signals and SFCM, the average

113


discrimination is 92.6% for the system to discriminate between 5 different facial gestures. The main reason to choose

these facial gestures are their generation source are far enough from Channel 2 and thus Channel 2 can be responsible to

extract affective measure without facial gestures interference. It was also shown that Channel 2 can mirror user

emotional state as the entropy of these Channel rises up when the level of difficulties was increased.

On another note, this training interface can be adapted for the users with disabilities from the user's neck down. Thus, it

can be a window to get to activities of daily life and these people can be more involved in the social activities. In our

futures steps, by using affective measures, the system can automatically adapt itself to the user status to decrease the

mental and cognitive stress while performing the operation.

ACKNOWLEDGEMENTS

We would like to thank Dr Christian Jones from USC-Australia for sharing his expertise on the area of

affective computing and also the help from eagerly volunteer participants is appreciated.

5. REFERENCES:

www.brainfinger.com, Accessed on 5 July 2009

Adams, N. (1996). A Study of The Effectiveness of Using Virtual Reality to Orient Line Workers in a Manufacturing

Environment. DePaul University, Chicago, IL.

Ang L., Belen E., Bernardo R A., Boongaling E. R., Briones G. H. and Coronel J. (2004). Facial Expression

Recognition through Pattern Analysis of Facial Movements Utilizing Electromyogram Sensors. IEEE

TENCON2004, 600-603.

Anon (1994). Immersive VR Tests Best. CyberEdge Journal, 4(6).

Anon (1996). VR in Industrial Training. Virtual Reality, 5(4), 31-33

Biopac (2009). http://www.biopac.com, Accessed on 5 July 2009.

.Bliss, J. P., Tidwell, P. D., & Guest, M. A. (1997). The Effectiveness of Virtual Reality For Administering Spatial

Navigation Training To Firefighters. Presence: Teleoperators and Virtual Environments, 6, 73-86.

Fernandez, R. (1997). Stochastic Modelling of Physiological Signals with Hidden Markov Models: A Step toward

Frustration Detection in Human-Computer Interfaces.

Firoozabadi, S. M. P., Oskoei , M. A., & Hu, H. (2008). A Human-Computer Interface based on Forehead Multichannel

Biosignals to Control a Virtual Wheelchair. Paper presented at the ICBME08.

Goldberg, S. L., & Knerr, B. W. (1997). Collective Training in Virtual Environments: Exploring Performance

Requirements for Dismounted Soldier Simulation. In R. J. Seidel & P. R. Chantelier (Eds.), Virtual Reality,

Training’ s Future? (pp. 41-51). New York: Plenum.

Healey, J., & Picard, R. W. (2000). SmartCar: Detecting Driver Stress. Paper presented at the ICPR.00, Barcelona,

Spain.

Kim, K., Yoo, J., Kim, H., Son, W., & Lee, S. (2006). A Practical Biosignal-Based Human Interface Applicable to the

Assistive Systems for People with Motor Impairment. IEICE Trans. Inf. & Syst., E89-D(10), 2644-2652.

Lintern, G., Roscoe, S. N., Koonce, J. M., & Segal, L. D. (1990). Transfer of Landing Skills in Beginning Flight

Simulation. Human Factors, 32, 319-327.

Loftin, R. B., Savely, R. T., Benedetti, R., Culbert, C., Pusch, L., Jones, R., et al. (1997). Virtual Environment

Technology in Training: Results from the Hubble Space Telescope Mission. In R. J. Seidel & P. R. Chantelier

(Eds.), Virtual Reality, Training’ s Future? (pp. 93-103). New York: Plenum.

Magee, L. E. (1997). Virtual Reality Simulator (VRS) for Training Ship Handling Skills. In R. J. Seidel & P. R.

Chantelier (Eds.), Virtual Reality, Training’ s Future? (pp. 19-29). New York: Plenum.

Mahoney, D. P. (1997). Defensive Driving. Computer Graphics World, 20, 71-71.

Mahlke S. and Minge M. (2006). Emotions and EMG Measures of Facial Muscles in Interactive Contexts. ( not

published ).

114


Mastaglio, T. W., & Callahan, R. (1995). A Large-Scale Complex Virtual Environment for Team Training. Computer,

28(7), 49-56.

Moertini V.(2002) ,Introduction to Five Data Clustering Algorithm, Integral, Vol. 7, No. 2, October

Nasoz, F., Lisetti, C. L., Alvarez, K., & Finkelstein, N. (2003). Emotion recognition from Physiological Signals for

User Modelling of Affect. Paper presented at the the 3rd Workshop on Affective and Attitude user Modeling,

Pittsburgh, PA.

Neimenlehto P., Juhola M. and Surakka (2006), Detection of Electromyographic Signal from Facial Muscles with

Neural Networks, ACM Trans. Applied Perception, Vol. 3, No.1, January 2006, 48-61

Oskoei M. A. and Hu H. (2007). Myoelectric control systems- A survey. J. Biomedical Signal Processing and Control,

vol. 2, issue 4, 275-294.

Picard, R. W. (1997). Affective Computing. Cambridge, MA: MIT Press.

Psotka, J. (1995). Immersive Training Systems: Virtual Reality and Education and Training. Instructional Science, 23,

405-431.

Priyona A., Ridwan M. , Alias A. , Atiq R., Rahmat O.K., Hassan A., and Ali M. (2003). Generation of Fuzzy Rules

with Subtractive Clustering, Universiti Teknologi Malaysia, Jurnal Teknologi, 43(D) Dis. 143–153

Regian, J. W., Shebilske, W. L., & Monk, J. M. (1992). Virtual Reality: an Instructional Medium for Visual-Spatial

Tasks. Journal of Communication, 42, 136-149.

Rezazadeh, I. M., Firoozabadi, S. M. P., & Hu, H. (2009). Facial Gesture Classification Using Multi-Channel Forehead

Bioelectric Signals for Human-Machine Interaction Applications. International Journal of Human Computer

Studies, Elsevier.

Schroeder, R. (1995). Learning From Virtual Reality Applications in Education. Virtual Reality, 1, 33-40.

Seidel, R. J., & Chatelier, P. R. (1997). Virtual Reality, Training’s future?: perspectives on virtual reality and related

emerging technologies. New York: Plenum Press

Simlog (Producer). (2004), Podcast retrieved from http://www.simlog.com/index.html.

Wakefield, R. R., O’Brien, J. B., & Perng, J. (1996, December 16-19). Development and Application of Real-Time 3D

Simulators in Construction Training. Paper presented at the the International Conference on Construction

Training, Hong Kong.

Wilson, B. H., Mourant, R. R., Li, M., & Xu, W. (1998). A Virtual Environment for Training Overhead Crane

Operators: Real-time Implementation. IIE Transactions, 30(7), 589-595.

115


CONSTRUCTION DASHBOARD: AN EXPLORATORY

INFORMATION VISUALIZATION TOOL FOR MULTI-SYSTEM

CONSTRUCTION

Cheng-Han Kuo, Master Student,

Computer-Aided Engineering Group, Civil Engineering, National Taiwan University, Taipei, Taiwan

[email protected]

Meng-Han Tsai, PhD Candidate,


[email protected]

Shih-Chung Kang, Assistant Professor,


[email protected]; http://www.caece.net/sckang

Shang-Hsien Hsieh, Professor,


[email protected]; http://myweb.caece.net/shhsieh/index

ABSTRACT: This research examines an interactive construction information presentation tool called

Construction Dashboard, designed to explore inter-dependent information visualization. Construction

Dashboard allows personnel to comprehensively compare the situations of operational conflicts of services to

ease the decision-making process. This research developed a virtual multi-system building project to work with,

and conducted a user test with 30 participants by offering 9 practical problem related questionnaires (3 level

tasks) to see participants' problem-solving performances in both presentation modes. The results of the user test

indicate that Construction Dashboard can effectively save time and increase the accuracy of work. This

interactive approach of information display offers a means to explore, compare, and recognize the underlying

constraints of construction issues in a more profession manner.

KEYWORDS: Information Presentation Tool, Inter-Dependent Information Visualization, Multi-System, MEP,

Construction Dashboard

1. RESEARCH BACKGROUND

Inefficiencies in the visualization method used to display interrelated project information and the prevalence of

traditional paper-based methods have made the precise demonstration and individualized information of project

activities a difficult task (Liston et al. 2000; Fischer et al. 2002). This research aims to solve two types of

problems specifically in multi-system construction of a building; the first is to promptly solve conflicts among

service systems to sequence the work programs, and the second is to easily comprehend complex information in

a multi-system construction project (Fard et al. 2006).

To achieve the goal, this research proposes an interactive presentation framework tool for multi-system

constructions. Interactivity relies on visual data representations in a framework with construction scenarios,

which can in turn trigger a possible solution to avoid operational conflicts for execution in the practical field.

This is achieved through interactive data visualization, supported by a graphical interactive framework

mechanism that enables a complete, direct and meaningful presentation.

The proposed framework includes two presentation approaches (Flexible Information Layout and Information

Display Matrix), and four interactive methods (Visual Data Exploration, Cross-Highlighting, Time Controller

and Information Extraction. Based on these approaches, we have implemented an information display tool for

construction of multiple service network systems in building, which is named Construction Dashboard.

Construction engineers can examine the construction information through direct manipulation and get

immediate feedback from Construction Dashboard, due to the dynamic interrelation of data. Therefore,

engineers can understand the interdependencies of construction information of different service systems, and

deal with the potential problems in advance. In this work, we address the problem of direct data interaction,

focusing particularly on the case where interactive presentation is most important.

117


2. INFORMATION IN MULTI-SYSTEM CONSTRUCTION

This research aims to focus different modes of information presentation in the construction process for

coordination of multi-system construction and understanding of massive interdependencies of construction

information. We propose four kinds of critical information in this respect:

• Temporal Information: The temporal information is an essential factor in construction ground that

is widely used in project planning and management. A project manager has to depend on temporal

information to control and monitor progress of a project with a view to completion by the scheduled

time. To represent temporal information, this research used the Gantt chart, as generally used in

construction management.

• Spatial Information: The spatial information includes geometric information of the assembly, route

and installation positions of the construction elements. To achieve high building performance in a

multi-system construction project, construction engineers usually use spatial information for

interference checking of distribution systems, paths, circuits, ducts, fittings, shapes and

configurations, feeders, panels, and so forth of MEP works to solve collision problems. The layouts

and positions of construction elements of a multi-service system are of a composite character as they

pass through, sustained or are placed closely to each other, or cross, overlap or clash in multiple

places. Consequently, conflict problems occur frequently in multi-system constructions, making

spatial information one of the most important forms of information for multi-system construction.

Presently, 3D models and rendering are widely used to represent spatial information. This research

followed these conventions.

• Hierarchical Information: The hierarchical information includes grouping manners, and the

hierarchical relationship of work items. It can help a construction manager understand the scope,

responsibility and organization of work items. In the multi-system constructions, a project manager

needs to clearly understand the classification and organization of each system in order to efficiently

control the project and thereby achieve high building performance. This research uses tree diagrams

because they effectively represent the hierarchical relationship and organization of information, and

are easily readable for less-experienced users.

• Relational Information: The relational information includes a number of relationships established

from the trial run of work items and system elements. In the trial run phase, systems usually need to

cooperate and mutually support each other. For example, a water supply system requires a power

system to provide power to run its motor. Similarly, AC systems are in principle dependent on a

power and water supply system. Hence, if the project manager cannot understand the relationships

between trial run work items and system elements, it will lead to project delays. We used a network

chart because its nodes and edges can represent the relationship of work items in an easily

comprehensible way.

3. PROPOSED PRESENTATION METHODS

Appropriate information presentation methods are the perfect cognitive process that can help users to browse,

explore, and compare information at pre-construction, construction and post-construction phases of a project.

Hence, this research proposed two presentation methods of multi-system construction:

• Flexible Information Layout: Flexible information layout can let users modify view-size and the

position of information according to its importance and weight. Therefore, users can easily discover,

explore, and compare particular information to concentrate on one or more specific piece of

information that they need. For these reasons, we adopted flexible information layout technology in

developing the presentation skill of Construction Dashboard.

• Information Display Matrix: Because of the limitations of monitor size and computer resolution, we

usually combined several monitors in a matrix form to display multi-information by keeping the

quantity and visibility of information the same. The Information Display Matrix can ensure that all

information has at least the same display area and links with each other; users can then freely move

information objects in any direction, such as up to down, left to right, or vice versa.

118


4. PROPOSED INTERACTION METHODS

Information interaction approaches can increase the visibility and readability of information so users can browse,

and diagnose information effectively. Hence, this research proposed four interactive approaches of multi-system

construction:

• Visual Information Exploration: In order to users can easily explore information, this research

proposes three kinds of visual information exploration: zooming, panning, and rotation from visual

information seeking mantra (Shneiderman, 1996).

• Cross-Highlighting: Cross-highlighting can represent the relationships between different information.

Therefore, it can help users in describing, explaining, and comparing information about a construction

project, leading to improvement in the decision making process (Liston and Fischer, 2000). For this

reason, this research proposed a cross-highlighting function to represent the relationships between

different information.

• Use of Time Controller: This research proposes a time controller, which integrates other

construction information. Users can use this time controller to manipulate temporal information to

understand the relationship among different kinds information. This research also introduces the use

of different colors that takes place with the operation of the time controller to represent differences in

construction status.

• Information Extraction: Information extraction approaches include information filtering,

information emphasizing, and information searching. Information filtering can let users hide

irrelevant information to help show only significant items. Information emphasizing can let users pick

up and display significant information, and at the same time hide unnecessary information.

Information searching can let users view relevant items that comply with keyword commands given

by users.

5. IMPLEMENTATION OF A CONSTRUCTION DASHBOARD

5.1 System Architecture

The overall composition of Construction Dashboard is illustrated in FIG. 1. It has a three-layer structure

comprising a user-interface layer, a data-process layer, and a data-storage layer. Each layer is composed of

major components or functions, which are represented by blocks. The arrows between each layer represent the

direction of communication or flow of data. In this design format, the user-interface layer is responsible for

providing functions so that users can manipulate and interact with elements on each view directly to present the

results in a visual way. The data process layer is responsible for extracting data from the database and producing

the view table needed to generate elements on each view based on user’s requirements. The data storage layer is

responsible for storing various construction data.

During software design, users can manipulate views and elements on each display directly, and get visual

feedback in the manipulating process simultaneously. The following paragraphs describe each feature of the

Construction Dashboard separately.

FIG. 1: Three Layer Structure of Construction Dashboard

119


• Views: The views are responsible for presenting construction information in a cognitive way. Each view

extracted required data from the view table to produce elements according to the information required to

display, and then presents and arranges the elements based on the layout algorithm of view.

• Manipulator: Users can manipulate the views and elements with two major functions of Construction

Dashboard. One is visual interactive functions and the other is cross-view functions. When a user uses

visual interactive functions, such as zooming and panning of view, the manipulator will maneuver the view

directly. When a user uses the cross-view functions, such as cross-highlighting, information extraction, and

information searching, the manipulator will search other views’ elements, which are using the same row of

the view table, and start the corresponding functions.

• View Table: The view table is responsible for providing data to each view. Each column of the view table

represents item-wise construction data, and each row represents property-wise construction information.

• Data Processor: The Data Processor is responsible for extracting data from the database to update the view

table to produce elements on each view, as per user’s requirements, and activate cross-view functions.

5.2 Information Presentation in Construction Dashboard

Construction Dashboard is designed with two information presentation skills, the flexible information layout,

and information display matrix. The following paragraphs explain each presentation function of the

Construction Dashboard separately.

• Flexible Information Layout: As shown in Fig. 2, all information layout views of Construction

Dashboard are floating windows. Hence, users can adjust the size and position of each view according to

the requirements of viewers.

• Information Display Matrix: As shown in Fig. 3, this research used four 24” LCD monitors and arranges

them in a matrix style, so that users can easily regulate information pages up to down, left to right and vice

versa.

FIG. 2: Flexible Information Layout

FIG. 3: Information Display Matrix

120


5.3 Information Interaction in Construction Dashboard

• Visual Information Exploration: As shown in FIG. 4, it can perform Zooming, Panning, and Rotation of

displays (Information views). As Construction Dashboard emphasizes vector graphics, the user can

minimize or enlarge elements of any view with a clear and smooth look, keeping the screen’s resolution or

DPI setting unchanged.

• Cross-Highlighting Technique: As shown in FIG. 5, when a user selects elements from any view of

Construction Dashboard, the elements will change its color to green, and other relevant elements of that

selected element will change their color to dark-blue. For example, when a bar of the temporal

information view is selected, the related 3D object of spatial view, the nodes of hierarchical view, and the

items of relational information view, will change their colors. Thus users can observe and understand the

interdependencies between construction elements easily.

• Time Controlling Maneuver: As shown in FIG. 6, after opening the time controller window, the element

of each view will change color to represent the construction status according to its construction start time,

finish time, and the current time of the time controller. Construction Dashboard uses three different colors

to represent the construction status of work items: gray represents the element that has not been started yet,

orange represents the element under construction, and magenta represents the element already completed.

• Information Extraction Role of Construction Dashboard: Construction Dashboard can perform three

functions i.e., information filtering, information emphasizing, and information searching, to exploit its

information extraction performance. Information filtering, as shown in FIG. 7, in the hierarchical

information view, is where the construction director can use a filtering function to hide irrelevant nodes

for the purpose of showing the significant node(s) as per user’s demand. Similarly, Construction

Dashboard, in another view of the display matrix, can also conceal or reduce particular element(s) through

a cross-highlighting function to show the importance of a particular element. Information emphasizing, as

shown in FIG. 8 is where users like to observe the relationship among nodes in the relational information

view. The users only need to put the mouse over the desired node, and then Construction Dashboard will

only show the relevant nodes. The brightness and colors of other irrelevant nodes will diminish. Thus

irrelevant nodes disappear from the screen. Information searching, as shown in FIG. 9, allows users to

select any list of items to search in the searching window, and then Construction Dashboard will highlight

the objects and relevant information in each view by using the same data.

FIG. 4: Visual Information Exploration

121


FIG. 5: Cross-Highlighting

FIG. 6: Time-Controlling Maneuver

FIG.7: Information Filtering

122


FIG. 8: Information Emphasizing

FIG. 9: Information Searching

6. EXAMPLE CASE OF A MULTIPLE SYSTEM CONSTRUCTION

We created an example case to validate the practicality of Construction Dashboard. We built a virtual building

project as an example case in a multisystem computer room, which has 154 elements in the 3D model and a

project duration of approximately 100 days. Here we intended to verify how effectively Construction Dashboard

can help users explore and understand relevant construction information. We also investigated whether users can

effectively identify the problematic work areas in a user test by exploring information from Construction

Dashboard. The details are described as follows.

In this example case, we refer to a project to build a 3D model with the help of a multi-system computer room in

the Faculty of Civil Engineering building at National Taiwan University. The example case involves a water

system, power system, air-conditioning system and computer system (FIG. 10).

FIG. 10: Example Case

123


7. USER TEST

7.1 Test Plan

As shown in FIG. 11, the test plan followed the 2×3×3 arrangement (2 is for exhibit manners, 3 denotes the tasks to perform and 3 is the three levels of questionnaires). All users needed to perform two exhibit manners; i.e. presentation through Construction Dashboard and presentation through paper-based media. In the test, users were required to perform three tasks, each of which included three levels of questions to answer. The following sections explain the test plan in more detail.

FIG. 11: User Test Procedure

7.2 Test Participants

There were 30 participants in the user test, comprised of 21 males and 9 females. Their ages ranged from

twenty-three to thirty-four years. The participants included 8 civil engineers and 22 graduate students from a

civil engineering background. All participants had studied construction management related courses.

7.3 Test Results

An α level of 0.05 was used for all statistical tests and analysis. The test results assessed how quickly and

accurately participants performed the task when using Construction Dashboard. Thus we measured the

efficiency and success rates of Construction Dashboard. They are summarized as follows:

• Efficiency: The statistical results, including mean value and standard deviation for five dimensions are

shown in FIG. 12. All of the tasks were considered significant (p<0.05) in t test. Users of Construction

Dashboard spent less time than required when using paper-based media.

• Success Rates: The success rates are shown in FIG. 13. From the figure, we can find that Construction

Dashboard results in higher averaging successful rate in eight out of nine testing tasks. We also used

statistical methods, t test and chi-square test, to see the difference between groups. We found that three of

the tasks, including Equipment-Coordination, Equipment-Projectwise and Function-Projectwise, reach

statistical significance.

124


FIG. 12: Test Results of Efficiency

FIG. 13: Test Results of Success Rates

8. FINDINGS FROM THE USER TEST

Overall, the test results were positive regarding the use of Construction Dashboard as a presentation tool for

interactive information presentation. From the user test we found conclusive evidence in four aspects:

• We used a matrix style for presentation of Construction Dashboard. Most users deemed it helpful when

comparing different information at a time, thus reducing the difficulty of memorizing more data. A minority

of users opined that they were tired of raising their heads to look over information.

• We used a flexible information layout for Construction Dashboard. All users agreed that this made their

125


work more convenient. The flexible information layout function is accorded with good presentation

phenomenon, and is compatible and favorable for presentation purposes that users might feel it is easy to

operate.

• All users agreed that the interaction mechanism to present interdependent information was better than

before. They felt that Construction Dashboard could improve the cognitive power required to catch

construction related information more precisely.

• In the test, we found that when users performed the paper-based presentation questioners first, most users

used the same method to answer the questions for the test of Construction Dashboard’s presentation. Most

forgot the function of Construction Dashboard in the test. Although we taught them and let them practice

before the test, the time of tutoring and practice was too short to remember some important functions of

Construction Dashboard.

9. CONCLUSIONS

We developed an interactive construction information display tool called Construction Dashboard on top of the

traditional paper-based mode of presentation. Based on this tool, users can explore, manipulate and compare

construction information through interactive visualization, and then make decisions in a more pragmatic way to

avoid collisions of construction systems. This research explored realistic information presentation and

interactive approaches to improve users' cognition of the relationship among critical construction information,

which enhances the communication of design intent to clients and helps avoid costly redesign onsite. We used a

multi-system computer room for our experiment, and designed a series of questions for our user test to compare

Construction Dashboard with the traditional presentation mode. In the test result, we discovered Construction

Dashboard can help users to explore and understand the inter-dependencies and underlying meaning of

information, thereby making it possible to solve problems more efficiently. Thus users are able to predict, solve

and answer questions more efficiently, after minimizing design coordination errors.

10. REFERENCE

Fard, M. G., Staub-French, S., Po, B., and Tory, M. (2006). “Requirements for a Mobile Interactive Workspace

to Support Design Development and Coordination,” Joint International Conference on Computing and

Decision Making in Civil and Building Engineering, Montréal, Canada, 14.-16. June 2006

Fischer, M., Stone M., Liston, K., Kunz, J., and Singhal, V. (2002). "Multi-stakeholder collaboration: The CIFE

iRoom," Proceedings CIB W78 Conference 2002: Distributing Knowledge in Building, Aarhus School of

Architecture and Centre for Integrated Design, Aarhus, Denmark, 12.-14. June 2002

Liston, K., Fischer, M. and Kunz, J. (2000). “Designing and Evaluating Visualization Techniques for

Construction Planning,” The 8th International Conference on Computing in Civil and Building Engineering,

Stanford, CA, USA.

Liston, K., Kunz, J., and Fischer, M. (2000). “Requirements and Benefits of Interactive Information Workspaces

in Construction,” The 8th International Conference on Computing in Civil and Building Engineering,

Stanford, CA, USA.

Shneiderman, B. (1996). “The Eyes Have It: A Task by Data Type Taxonomy for Information Visualization,”

Proc. Visual Languages, pp. 336-343.

126


COMPUTER GAMING TECHNOLOGY AND POROSITY

Russell Lowe, Senior Lecture in Architecture,

University of New South Wales, Faculty of the Built Environment;

[email protected] , www.russelllowe.com

Richard Goodwin, Professor of Fine Art,

University of New South Wales, College of Fine Art;

[email protected] , www.richard-goodwin.com

ABSTRACT: In 1996 artist-architect Richard Goodwin coined the term "Porosity". Porosity describes the publicly

accessible spaces within privately owned parts of the city. Any mixed use building is necessarily Porous; for

example, clients must be able to visit their dentist's surgery on the 14th floor, their lawyer on the 5th floor, or a

restaurant on the roof. A buildings Porosity is a measure of the quantity and quality of pathways to a given

destination (Goodwin 2006).

More recently, the growing list of urban mapping projects suggests that there is an urgent need for a deeper

understanding of the dynamic relationship between public access and the occupiable spaces of the city (see Reades

et al 2007, for a representative range of these, C. Nold's work is worth a special mention). The Porosity of a

building is an excellent example of the dynamic relationship between people and the built fabric of the city. Due to

the manual data gathering techniques employed, the first incarnation of the Porosity maps were only able to create

a 'snapshot' of the buildings selected. To understand how the Porosity of a specific building might change over time

the mapping process would need to be automated.

The questions that initiated this research were, "could Porosity be represented in real time? What should that

representation look like? And can the combination of computer gaming technology and environmental sensors

automate the representation of Porosity?"

In response to these questions the authors have developed a prototype that translates the movement of a person in

the real world into the virtual environment of a computer game; note the pedestrians' participation is entirely

passive (i.e. they are not knowingly playing a computer game, they are simply going about their business). The

movements of a Non-Player Avatar, standing in for the pedestrian, are then represented with a range of textures,

geometries and behaviors. (The external sensor that is being used to demonstrate proof of concept is the Nintendo

Wii Balance Board, employing a custom script to interface with the PC). The authors call these representations of

movement and time 'Porosity Lenses'. Their development draws from Goodwin's Porosity Index but, significantly,

construct it in real time. In one lens the movement of the avatar constructs a facsimile of a space as sensors

passively capture a person's movement through the real one.

Finally the paper compares the lenses developed with recent representations of movement over time to highlight

strengths and weaknesses of the approach.

KEYWORDS: Porosity, Computer Games, Sensors, Representation, Mapping.

1. INTRODUCTION

A growing list of urban mapping projects suggests there is an urgent need for a deeper understanding of the dynamic

relationship between public access and the occupiable spaces of the city (see Reades et al 2007, for a representative

range of these, C. Nold's work is worth a special mention). In many cases these projects represent dramatically

changing patterns of use, mobility, and security. The term "Porosity", coined by Richard Goodwin, describes the

publicly accessible spaces within privately owned parts of the city. With the support of an Australian Research

Council Discovery Grant from 2003-2005 Goodwin and his research team mapped these "Porous" spaces within the

Sydney CBD. The results suggest new opportunities for pedestrian movement through the city. In contrast to many

127


of the urban mapping projects cited above the Porosity maps are fully three dimensional. By recording the member

of the public's duration of stay they also capture the dimension of time. However, due to the manual data gathering

techniques employed, the first incarnation of the Porosity maps were only able to create a 'snapshot' of the buildings

selected. To understand how the Porosity of a specific building might change over time the mapping process would

need to be automated. This prompted questions such as can Porosity be represented over time, and ideally in real

time? What should that representation look like? Can the combination of computer gaming technology and

environmental sensors automate the representation of Porosity?'

In this paper the authors describe a new way to map the Porosity of a building by modifying an off-the-shelf

computer game, Unreal Tournament 3 (UT3) by Epic games, and using sensor controlled "Non-Player Avatars". In a

typical single player computer game the player knowingly controls an "Avatar" (which is the players embodiment

within the virtual world) and may compete against or be assisted by Non-Player Characters (NPC's) which are

controlled by the computer game's artificial intelligence. In a multiplayer game the player is usually competing

against or being assisted by Avatars that are knowingly controlled by other real people. In contrast to these typical

situations the authors have created a prototype where a real pedestrian's presence in the virtual environment is

entirely passive i.e. they are not using the computer or knowingly playing a computer game, they are simply going

about their business. This also contrasts Gemeinboeck et al's (2005) approach where the "spatial evolution …

unfolds in the mutual interplay between the participant and the virtual opposite." To clarify this distinction the

authors have coined the hybrid term "Non-Player Avatar" or NPA. The movements of an NPA are driven by sensors

recording a pedestrian's movement within a real environment and simultaneously traced in virtual space and time

with a range of textures, geometries and behaviors. The authors call these representations of movement and time

'Porosity Lenses'.

The Porosity Lenses are designed to facilitate analysis, by Avatars within the virtual environment, of NPA

movement from many different points of view thereby taking advantage of Sun et al's (2007) research that notes the

importance of three dimensional space and point-of-view in shaping human behavior in urban spaces. In an

extension to Sun el al's work the authors propose that if three dimensional space and the first person point-of-view is

important in shaping human behavior in urban spaces then they may be equally important in understanding and

analyzing human behavior in these situations.

This paper will describe the theoretical context, design and development of the Porosity Lenses, while comparing

and contrasting them to recent representations of movement over time. In one example the movement of an avatar

generates three dimensional building blocks that construct a facsimile of the pedestrian's environment in real time as

sensors record their movement through it. It will also describe the development and preliminary testing of a

prototype sensor solution that uses off the shelf computer gaming hardware.

As noted above there are many examples of projects that map the movements of people in cities and the developers

of Halo 3 (Bungie Studios in collaboration with Microsoft) have mapped and analyzed over 3,000 hours of game

play (Thompson 2007). The research presented here uses off the shelf computer gaming software and hardware with

a view to combining and extending the two approaches.

2. THE POROSITY STUDIO.

In 1996 artist-architect Richard Goodwin established the Porosity studio within the College of Fine Arts (COFA) at

the University of New South Wales. Goodwin notes that the public space of the city doesn't end at the building

envelope; that any mixed use building requires access by the public and is necessarily porous.

For example, clients must be able to visit their dentist's surgery on the 14th floor of a building, their lawyer on the

5th floor, or a restaurant on the roof. A buildings Porosity is a measure of the quantity (and quality) of pathways to a

given destination (Goodwin 2006). The primary concern of Goodwin's research was the amount of time that the

Porosity researcher could spend within a privately owned building without detection. After comprehensive

fieldwork detailed three dimensional maps of three major zones within the Sydney Central Business District (CBD)

were produced. Architectural data on the buildings within those three zones, combined with the results of the field

work, gave each building a qualitative Porosity Index. In the original Porosity Index Goodwin cited orientation,

duration of stay, adjacency to lifts, stairs and other distinctive architectural qualities as factors that contribute to a

building's Porosity Index.

128


Goodwin proposed that reading these Porosity Indexes could give planners and architects direction as to the way in

which new linkages may be made which enhance the public space in the city. But from a slightly more sinister point

of view the Porosity Index's can also measure and qualify the dilemmas of security versus access in relation to

public and private space. In other words the Porousness of a building relates to the ease by which a building might

be accessed and evacuated; the irony here is that high levels of Porousness would seem to facilitate both.

3. THE APPROPRIATENESS OF USING COMPUTER GAMING TECHNOLOGY TO REPRESENT POROSITY IN REAL TIME.

Rather than being fully spatial many urban mapping projects still represent the city in two dimensions. A few

projects supplement those dimensions with other, non-spatial, dimensions such as information about the users

itinerary (E. Polak's "Amsterdam RealTime: Diary in Traces") and physiological responses (C. Nold's Biomapping

project, 2004 - ongoing). In "The Language of New Media" (2000) Lev Manovich says that "along with providing a

key foundation for new media aesthetics, navigable space has also become the new tool of labor." He goes on to say

that "the 3-D virtual space combined with a camera model is the accepted way to visualize all information".

Demonstrating the pragmatic advantage of visualizing information in this way Sun et al (2007) recognized the

importance of point-of-view in shaping human behavior in urban spaces. By using a head cave and three

dimensional virtual environments they found that some assumptions about human behavior in urban spaces could be

challenged. It's interesting to note here that their research utilized an environment that was "designed to be

something like a first person shooting game, such as DOOM." In contrast the authors did not use something like a

first person shooting game, they actually used an off the shelf computer game. So why take Sun et al so literally and

use an off the shelf technology designed in the first place for entertainment? The answer lies in the underlying

sophistication of computer games and, more recently, their versatility regarding modification; not to mention the

widespread encouragement by game developers and large "modding" communities that users create custom game

dynamics and content.

Microsoft Research wrote in 2005 that computer game technology "pushes the technology envelope". Also

recognising this, the game developer "Virtual Hero's" (the developer of the groundbreaking simulation/marketing

tool "Americas Army") has recently licensed the UT3 game engine to develop an urban training simulation called

Zero Hour: Americas Medic. Illustrating the growing institutional acceptance of repurposing entertainment

technology Virtual Hero's are also working in collaboration with the American Department of Homeland Security.

In 2006 Price provided a useful summary of the recent "deployment of game engine technology" for education and

training applications, noting that their use in such ways has occurred only in the "last few years". In 2000

Bouchlaghem et al noted that "the benefits and applications of virtual reality (VR) in the construction industry have

been investigated for almost a decade" but that "the practical implementation of VR in the construction industry has

yet to reach maturity owing to technical constraints". Ten years on from Bouchlaghem et al's study we find that

industry heavyweight Autodesk has recently become a member of the "Integrated Partners Program" with EPIC

games and, as Author 1 has mentioned in a previous paper, the use of computer gaming technology in education

reflects moves in practice by large firms such as Texas based HKS and small by Lara Calder Architects, Sydney. In

a move that represents an alternative way forward to their main Building Information Modeling (BIM) competitor

(Autodesk's Revit) Graphisoft's Archicad has incorporated a real time interactive engine in its latest release. While

the authors applaud the initiative they remain skeptical that Graphisofts resources and culture will facilitate their

engines comprehensive development (or importantly, many alternative developments). The use of VR technologies

in the construction industry is still not mature, but many of their technical constraints have been overcome …

somewhat ironically, by the entertainment industry.

In the discussion relating to human behavior in public spaces Sun et al state their assumption that "from the set of

architectural clues in sight, the human selects the one with the highest priority and performs a related strategy"

(2001). To reiterate, in the context of the authors project as humans perform navigational strategies within a real

environment NPA's mirror them within a virtual environment. The Porosity Lenses described below are mechanisms

that add persistent traces of movement to the "set of architectural clues in sight" so that Avatar analysts can develop

an understanding of the movement of NPA's; and by extension the pedestrians that are driving them. By utilizing

computer games the Avatar analysts can take advantage of the "positive benefits of video game play" that include

"spatial visualization and mental rotation" (Rosser et al, 2007).

129


4. DESIGNING 'POROSITY LENSES' USING UNREAL TOURNAMENT 3.

Representing one of the latest generation of computer gaming technologies, Unreal Tournament 3 (UT3)

incorporates an incredibly comprehensive tool set and has the support of a large game moding community (see the

forums at www.3dBuzz.com and http://forums.epicgames.com/forumdisplay.php?f=335 for example). For these

reasons UT3 was chosen as the computer game medium within which to develop the initial functional prototypes of

the Porosity Lenses. The UT3 world editor, UnrealEd, "is a suite of tools for working with content in the Unreal

Engine. At the core, it is used for level [virtual environment] design; but contained within are editors and browsers

for importing and manipulating content for your game project (EPIC Games)." The toolsets used to design and

develop the Porosity Lenses include UnrealKismet, Matinee, Cascade, the Material and Static Mesh Editors. Many

of the toolsets require information generated in third party software (textures or geometry for example) and from

other toolsets within the editor itself. While this interdependence adds to the complexity of modifying the game it

does provide many opportunities to link different types of parameters and contributes to the sophistication of the

interactivity.

For the first Porosity Lens the UnrealKismet toolset was used in conjunction with the Cascade toolset to attach,

detach and control the emission of a sprite particle emitter that was attached to a NPA (a sprite is a 2d surface that

always faces the player). UnrealKismet "allows non-programmers to script complex gameplay flow in level. It

works by allowing you to connect simple functional Sequence Objects to form complex sequences (EPIC Games)."

In this and the following examples visual scripting was used to create a mechanism for interactivity that didn't exist

previously within the UT3 game. The result is that as the NPA moves around the environment it leaves a trail of

translucent squares that traces its movement through space and time; much like the breadcrumbs left by Hansel and

Gretel in the well known fable, figure 1. The custom material applied to the sprite contains a variable opacity-

parameter so that the translucency can be adjusted to balance between the clarity of the avatars path and the density

of its representation. The density of the path at any one point represents the overlaying of multiple translucent sprites

that build opacity and represent the duration spent at that point; a key factor to understanding Porosity.

Figure 1. Shows the results of the Kismet sequence created by the authors that controls the relationship between a

particle emitter and an NPA. The sprite is a translucent red square which is emitted at a rate of 10 instances per

second. The effect is much like the breadcrumbs left by Hansel and Gretel in the well known fable but in this case

records both the path taken and the duration spent at any point along it. Avatar analysts are also able to

dynamically control the translucency of existing structures within the environment.

Extending from the notion of parametric interdependence mentioned above, every element within the game

environment contains parameters that can be customized. Identifying parameters and referencing them within

Kismet enables many to be changed over time. These changes can be pre-scripted (much like conventional

animation) or can occur in real-time. For the second Porosity Lens a Kismet sequence was designed that modified a

130


scalar parameter which controls the opacity channel of a material applied to a rectilinear block within the

environment. The collision properties of the block are set to register "touch events" with NPA's but will not collide

with them physically, so that they do not impede their progress. Every time a block is touched by an NPA its opacity

drops by 0.2 (where 1.0 is opaque and 0 is transparent). The sequence continues reducing the resulting opacity, upon

subsequent touch events, until it reaches zero; i.e. until the material is completely transparent. Figure 2. shows a

simple arrangement of corridors filled with the blocks.

Figure 2. Shows the blocks in a corridor. Their opacity is controlled by a Kismet sequence linked to one of their

materials properties, each time a "touch event" occurs the opacity of the block is reduced by 0.2, until the block is

completely transparent. This gives the impression of the NPA or NPA's slowly carving out the space that they

occupy.

Multiple touches by one or more NPA's give the impression of their movement slowly "carving out" the space of the

corridors. One can see that as long as walls, floors, ceilings and doorways limit the pedestrian's movement in their

real environment there would be no need to represent them within the virtual environment; the presence or absence

of rectilinear blocks can perform this role. The authors imagine an environment totally filled with these blocks in a

complete 3d matrix. The virtual representation of space that is occupied by pedestrians in a real environment would

become clear as the NPA paralleling their movements traveled through it.

The third Porosity Lens adopts the additive approach of the Hansel and Gretel Lens with the 3d geometry of lens

two. In contrast to the Hansel and Gretel Lens however the mesh is only emitted when the avatar "carrier" is

moving; i.e. it records position rather than position and duration. The emitter could be set to emit mesh particles

constantly, but the additional processing required by 3d geometry over a simple sprite slows the computer

significantly when count rises into the thousands. At the current stage of its development the mesh elements do not

collide with the NPA. Ultimately the authors intend that the mesh will collide with the footsteps of the NPA so as

the pedestrians negotiate a real environment they passively construct a version of it beneath their feet in a virtual

environment. See figure 3.

In the three lenses described above two representational strategies are employed; the sprites and tiles are additive,

the rectilinear blocks are subtractive. Further, each of the prototypes utilizes arbitrary rectilinear shapes or geometry

and the textures used are homogonous. As working prototypes they demonstrate that the answer to the question "can

Porosity be represented in real-time?" is yes, but there is clearly room for improvement. The following section looks

at three examples of urban mapping projects that highlight strengths, weaknesses and suggest possible directions

regarding future development for the Porosity Lenses.

131


Figure 3. Real-Time construction of an environment. Blocks emitted by the NPA as they move around an empty

virtual environment would construct a facsimile of the pedestrians real environment. Note the image above shows

blocks leaping over ones previous laid avoiding an intersection; in those cases the NPA literally leaped over the

earlier path. This demonstrate that environments that change in the vertical dimension (stairs, ramps, etc) are able

to be replicated .

5. WHAT SHOULD THE POROSITY LENSES LOOK LIKE?

In the following examples the authors critically examine recent efforts by various researchers to represent the

movement of people through space.

In C. Nold's work, seen at www.biomapping.net, "participants are wired up with an innovative device which records

the wearer's Galvanic Skin Response (GSR), which is a simple indicator of the emotional arousal in conjunction

with their geographical location. People re-explore their local area by walking the neighbourhood with the device

and on their return a map is created which visualises points of high and low arousal." The resulting maps of

Stockport, Greenwich and San Francisco are two dimensional. The map for San Francisco uses stacked red disks and

at first appearance bears some similarities with the sprite based Porosity Lens. This similarity is short lived however

as the colour intensity isn't built up in layers but comprises arbitrary steps on a scale; each disk is opaque. A sample

video on www.biomapping.net shows a three dimensional structure built over Google Earth.

Figure 4. Screen capture from www.biomapping.net showing C. Nold's mapping of "emotional arousal in

conjunction with geographic data." The effect is similar to a line graph, folded over a surface in Google Earth. Note

the horizontal surface is completely flat and that any rise in elevation could confuse the reading of data in the z-axis.

132


The line in the xy plane traces the participant's movement through the environment while the z-axis is used to

represent emotional arousal. The effect is similar to a line graph, with solid fill beneath, which is folded so that it

might stand up unsupported. Figure 4. The height in the z-axis plays the same role as the density of the Hansel and

Gretel sprite trail, but this approach would quickly become confusing if the ground plane was not completely flat

(i.e. a point on the terrain, or within a building, with a higher elevation could give the impression that the experience

there is more intense). Expressing the data as a cross section area perpendicular to the direction of movement may

alleviate that confusion and suggests a future direction of research for the authors.

E. Polak et al's project Amsterdam RealTime tracks people equipped with GPS enabled devices in real time and

projects the resulting lines onto a black background. The map is 2 dimensional but expresses a direct precursor to a

key strategy of the Porosity Lenses (using people movement to build an environment); as explained by the Waag

Society "this map did not register streets or blocks of houses, but consisted of the sheer movements of real people."

The time lapse animation shown here: http://www.waag.org/project/realtime shows the map glowing and pulsating

at points where multiple pathways cross. The representation of intensity is unmistakable qualitatively but vague in a

quantitative sense. This may be a strength and weakness of all strategies that rely on multiple 2 dimensional layers.

Bungie Studios the developers of Halo 3 (in collaboration with Microsoft) have developed tools to extract gameplay

data so they could map and analyze over 3,000 hours of game play (Thompson 2007). In one example "player

deaths [are] represented in dark red on [a] 'heat map' of the level". Alison Mealey manipulates a similar approach to

create portraits by recording and representing the movements of NPC's in the game UT2004 (Petersen 2005).

Another example from Bungie shows "superimposed locations of about 30 testers after half an hour of gameplay";

players left different "coloured dots showing player location at five-second intervals (each colour is a new time

stamp)". When the dots were clustered by colour it demonstrates that "players were moving smoothly through the

map". Currently the Porosity Lens's don't implement a similar facility; not only would it show consistency of

movement through an environment it would also confirm the direction of that movement. Once again these

examples are 2 dimensional, but the final example from Bungie shows the importance of understanding actions from

a player's perspective in 3 dimensions. In this example Bungie's analysts noted a high rate of "suicides" in a

particular area of a map; the "heat map" would show them the location but the action in this case was not shaped

environmentally but locally. "The players were firing the tank's gun when its turret was pointed toward the ground,

attempting to wipe out nearby attackers. But the explosion ended up also killing (and frustrating) the player"

(Thompson 2007). In this example 2 dimensional orthographic views and real-time spatial experience combine to

give a more complete representation of the causes and effects of a user's navigation through an environment. See

Figure 5.

Figure 5. On the left is a 'Heat Map" showing the number of player deaths at particular points within the

environment. The map on the right shows specific colours marking player location at five-second intervals. This is

one strategy for confirming direction of movement; as well as clearly marking trends and those opposing them.

Within these examples three major issues arise; the qualitative vs quantitative advantages of utilising Colour

Intensity and Size as mechanisms to represent duration of stay in any one place; the obscuring of data by either

subsequent entries by the same person or by entries from another person; and the representation of the "activity

workspace" (Mallasi, 2004). The obscuring of data is seen in Nold's work (Figure 6) and in the authors (Figure 4)

and would be a consideration in the development of a cross sectional area representation mentioned above. Currently

the authors are able to adjust the translucency of elements within the UT3 environment in real time; extending this

functionality to the Porosity Lenses themselves may prove beneficial.

133


In a paper regarding construction activities' workspace conflicts Mallasi (2004) sees that "to specify the workspace

requirement in a dynamic way, while satisfying a set of spatial dynamics and change of workplace over time

intervals, is a difficult problem". To mitigate this problem in his research he uses a technique that represents the

activity workspace in a series of 3dimensional boxes. Each box defines a workspace (such as above, below, or

surrounding) that is a "generic capture of different workspace requirements." While the second Porosity Lens

similarly utilises rectilinear blocks it indexes them to the size of the construction worker him or herself. By doing

this a more fine grained understanding of the activity workspace would result (see Figure 2).

6. THE NINTENDO WII "BALANCE BOARD" AS AN EXTERNAL SENSOR LINKING PEDESTRIAN MOVEMENT TO UT3 NON PLAYER AVATAR MOVEMENT; AUTOMATING THE COLLECTION OF DATA.

In a typical computer game the avatar translates the actions of real people into the virtual environment.

Conventionally one controls their avatar directly and might use a computer keyboard and mouse or gamepad to do

so. In addition the computer controls various other characters within the game environment with artificial

intelligence. A third category, sensor controlled avatars, has recently emerged. These avatars are controlled by

sensors that pick up the movement of a person in a real environment and translate it to a virtual environment.

Groenda et al (2005) use tracking and Motion Compression to allow the exploration of "an arbitrarily large virtual

environment while the user is actually moving in an environment of limited size". While Motion Compression

would be useful for play in gaming halls or at home it takes this tracking technology in the opposite direction to the

authors' project that uses sensor controlled avatars for mapping real spaces. In other words, while Gronenda et al

saw that the virtual environment would be "limited to the size of the user's real environment" the authors see this as

an opportunity to map the limits of a real environment by tracing pedestrian movements through it.

Devices for tracking people through environments include GPS (Global Positioning Systems) INS (Inertial

Navigation Systems) and Radio Frequency (RF) based positioning systems. Due to the particular challenges of

urban environments none of the systems listed above offer a comprehensive solution. A hybrid system is required.

Allen et al note the "widespread availability" of low-cost computer game peripherals and see an opportunity to

"adapt technology designed for the entertainment industry and develop low-cost measurement tools for use in

clinical science and rehabilitation engineering (2007)." A major advantage for Allen et al is to break free from the

limits of the clinic; a key limit being expensive equipment. For the author's the major advantage is that with

"widespread availability" ultimately there may be a good chance that a pedestrian might already be carrying the

sensor we could use to track them.

With the notion of repurposing off the shelf computer game peripherals in mind the authors tested a Nintendo Wii

"Balance Board" that employed a custom script to interface with a laptop computer. The first version of the code

that connected the Balance Board with the PC was written by Nedim Jackman as a part of his undergraduate degree

in Computer Science. Jackman originally wrote the software to "measure the deterioration of aged people's balance

(Schwarts 2009 in conversation with the authors)." Jacob Schwarts, a Masters student studying with author 1,

worked with Jackman to adapt the code so that it translates motion on the WiiBoard to a set of configurable

keyboard signals. See "Jackman" in the references for a link to the code.

Both Schwarts and the authors have used the first version of the code to control the movements of avatars within

UT3; Schwarts to design and demonstrate his graduation project and the authors to create a three dimensional real-

time map of a person's physical movement within a virtual environment. In the first iteration the test subject's

movement is very limited; leaning forward/backwards/left/right replaces taking actual steps. While this represents an

alternative way to interact with the computer (i.e. not a traditional keyboard or game pad, Jefery Shaw's Legible City

1989-1991 is probably the most well know early example of this) it doesn't capture the act of walking passively.

Subsequently the authors worked with Jackman to enable the connection of up to 7 balance boards with the PC. A

video clip (www.russelllowe.com/publications/convr2009/convr2009.htm ) shows author 1 walking forward across

three boards with his movement being translated to the NPA in real time. The second part of the clip shows the

author (and NPA) walking to the left. These clips demonstrate that the steps by the author in the physical

environment produce a related amount of steps by the NPA in the virtual environment. Demonstrations on

www.youtube.com show a Nintendo Wii controller (youtube, 2009) being used to interface with Half-Life 2 (a first

person shooter game in many ways similar to UT3) and implicate a further related area for investigation i.e. the

134


integration of multiple gyroscopic devices. While these devices do not represent the ultimate solution they do

suggest computer game peripherals could play a role in it.

In contrast to GPS based systems that represent a 'collaboration' between sensors mounted in the environment and

sensors carried by pedestrians INS systems record the pedestrian movements independently. Rizos et al (2008)

present the notion of "bridging … GPS gaps" with INS systems which overcomes a significant drawback with INS

systems; "sensor errors that grow unbounded with time". When a player uses the Wii controller to manipulate an

avatar in Half-Life 2 cumulative error is overcome by constant adjustments made by the player in response to their

avatars position compared to their desired position. In the case of the NPA no such adjustments can/are be made (the

pedestrian sees neither the avatar nor environment). The construction site, by very definition, is in constant flux …

the physical environment may not exist to support sensors at one stage of construction and then may make GPS

based systems ineffective at a subsequent stage. Strategies for incorporating sensors within buildings as they are

constructed are necessary.

7. CONCLUSION AND FUTURE WORK.

Richard Goodwin's Porosity Project contributed to urban mapping in two very important ways; it recognised that

public spaces don't end at the envelope of a building and by extension it understood that navigating the city is a 3

dimensional proposition. A key factor of a buildings Porosity is the amount of time a person can spend in different

parts of a building; and this duration changes over time.

This research finds that it is possible to represent Porosity in real-time and that an advantageous medium to use to

achieve this is computer gaming technology. Extending from Sun et al the Porosity Lenses add persistent traces of

movement to the "set of architectural clues in sight" that analysts would be able to use to understand pedestrian

movement and space usage within an urban environment. By using computer games the analysts can take advantage

of the "positive benefits of video game play" that include "spatial visualization and mental rotation" (Rosser et al,

2007). The computer game UT3 was chosen to construct these prototypes because it represents one of the latest

generation of computer gaming technologies, it has a comprehensive and interconnected toolset, and the support of a

large game modding community. The Porosity Lenses develop additive and subtractive strategies that have grown

out of an examination and criticism of recent urban mapping projects. In this examination three major issues arise;

the qualitative vs quantitative advantages of utilising Colour Intensity and Size as mechanisms to represent duration

of stay in any one place; the obscuring of data by either subsequent entries by the same person or by entries from

another person; and the representation of the "activity workspace" (Mallasi, 2004). Both mechanisms have strengths

and weaknesses and further work is required to create a hybrid or develop new alternatives. By utilising a base unit

of workspace indexed to the construction worker, which follows them and build's (or carves as the case may be) a

total model of their space use over time, a more fine grained understanding of the activity workspace would result.

To represent Porosity in real-time first one must collect the data in real-time. Allen et al note the "widespread

availability" of low-cost computer game peripherals and with this in mind the authors the authors sought to extend

the modified off the shelf software approach to include hardware. With Nedim Jackman the authors connected 3

Nintendo Wii balance boards to a PC and were able to passively control an Avatar (now a Non Player Avatar, or

NPA). This demonstrates proof of concept, but it is by no means a complete solution; future work will look at

repurposing the Wiimote, Cellphones and Wireless Motes. Finally, while Gronenda et al saw that a virtual

environment would normally be "limited to the size of the user's real environment", and developed "Motion

Compression" to circumvent those limits, the authors see restricting the player to their real environment as an

opportunity to map the limits of that environment by tracing pedestrian movements through it.

8. REFERENCES.

Allen, D. Playfer, J. Aly, N. Duffry, P. Heald, A. Smith, S. And Halliday, D. (2007) "On the Use of Low-Cost

Computer Peripherals for the Assessment of Motor Dysfunction in Parkinson's Disease - Quantification of

Bradykinesia Using Target Tracking Tasks." In IEEE Transactions on Neural Systems and Rehabilitation

Engineering, Vol. 15, No 2. June 2007.

Author 1 (2008) reference regarding large Architecture firms using UT3.

135


Bouchlaghem, N, Khosowshahi, K. White, J (2000) "Virtual reality as a visualisation tool: benefits and constraints."

CIDAC Special Issue on Visualisation in Architecture, Engineering and Construction. Volume 2, Issue 4.

EPIC Games, Mod Community page: http://udn.epicgames.com/Three/UT3ModHome.html , recovered 14 January

2009

Autodesk press release: http://www.epicgames.com/press_releases/autodesk.html

HKS press release: http://www.hksinc.com/news/2007_10_HKS_Licenses_Unreal.htm

Gemeinboeck, P, Blach, R (2005) "Interfacing the Real and the Virtual: User Embodiment in Immersive Dynamic

Virtual Spaces." In Learning from the Past a Foundation for the Future [Special publication of papers

presented at the CAAD futures 2005 conference held at the Vienna University of Technology / ISBN 3-

85437-276-0], Vienna (Austria) 20-22 June 2005, pp. 171-180

Goodwin, R (2003-2005) Australian Research Council Discovery Grant, Project ID: DP0346062.

http://www.arc.gov.au/

Goodwin, R, McGillick, P, Helsel, S., Tawa, M., Benjamin, A., Wilson, G., (2006) Richard Goodwin: Performance

to Porosity, Craftsman House, an imprint of Thames and Hudson, Australia

Groenda, H (et al) 2005, Telepresence Techniques for Controlloing Avatar Motion in First Person Games, Maybury,

M. et al (Ed's.) INTETAIN 2005, Springer-Verlag Berlin. LNAI 3814, pp44-53

Hansel and Gretel. Fable. http://en.wikipedia.org/wiki/Hansel_and_Gretel recovered 22 January 2009.

Jackman, N (2009) http://code.google.com/p/wiiboard-simple/downloads/list recovered 12 January 2009.

Lara Calder Architects, Sydney: http://www.laracalderarchitect.com.au/

Manovich, L (2000) "The Language of New Media". MIT Press, Cambridge, MA

Microsoft Research, 2005, Computer Gaming to Enhance Computer Science Curriculum.

http://research.microsoft.com/ur/us/gaming/ComputerGamingToEnhanceCSCurriculum.doc recovered

November 2008.

Nintendo: Wii Balance Board http://www.nintendo.com.au/wii recovered 4 February 2009.

Nold, C (2004 - Ongoing) http://www.biomapping.net/ recovered November 2009.

Petersen, T (2005) "Generating Art from a Computer Game. An Interview with Alison Mealey"

http://www.artificial.dk/articles/alison.htm recovered 13 July 2009.

Polak, E (2002) "Amsterdam RealTime: Diary in Traces" For the exhibition "Maps of Amsterdam 1866-2000 at the

Amsterdam City Archive. http://project.waag.org/realtime/en_frame.html recovered 9 January 2009.

Price, C (2006) "A Crisis in Physics Education: Games to the Rescue!" in ITALICS, Innovation in Teaching And

Learning in Information and Computer Sciences, Volume 5 Issue 3.

http://www.ics.heacademy.ac.uk/italics/vol5iss3/price.pdf recovered 1 Feb 2009.

Reades, J, Calabrese, F, Sevtsuk, A, Ratti, C, (2007) Cellular Census: Explorations in Urban Data Collection, IEEE

Computer Society, Pervasive Computing, Vol. 6, No 3, July-September.

Rizos, C, Grejner-Brzezinska, D.A, Toth, C.K., Demster, A.G, LI, Y, Politi, N, and Barnes, J. (2008). A hybrid

system for navigation in GPS-challenged environments: Case study. 21st Int. Tech. Meeting of the Satellite

Division of the U.S. Inst. of Navigation, Savannah, Georgia, 16-19 September, 1418-1428.

Rosser, J, Lynch, P, Cuddihy, L, Gentile, D, Klonsky, J and Merrell, R (2007) "The impact of Video Games on

Training Surgeons in the 21st Century. Archives of Surgery, Vol. 142 No. 2, http://archsurg.ama-

assn.org/cgi/content/full/142/2/181 recovered 13 January 2009.

Shaw, J (1989-1991) "Legible City" http://www.jeffrey-shaw.net/html_main/frameset-works.php3 recovered 14 July

2009.

136


Sun, C, De Vries, B, Dijkstra, J, (2007) Measuring Human Behaviour Using a Head-Cave. CAADFutures, pp. 501-

511.

Thompson, C (2007) Halo 3: How Microsoft Labs Invented a new Science of Play, Wired Magazine, issue 15.09

available online here: http://www.wired.com/gaming/virtualworlds/magazine/15-09/ff_halo recovered 13

July 2009.

Townsend, A (2000), Life in the Real-Time City: Mobile Telephones and Urban Metabolism. Journal of Urban

Technology, 7:2, 2000, pp85-104

YouTube (2009) Search "WiiMote playing Half-Life 2 on a computer".

http://au.youtube.com/watch?v=asY_I8y6C0M recovered 16 January 2009.

137


VIRTUAL REALITY USER INTERFACES FOR THE EFFECTIVE EXPLORATION AND PRESENTATION OF ARCHAEOLOGICAL SITES

Daniel Keymer, Mr,

Department of Computer Science, University of Auckland;

[email protected]

Burkhard Wünsche, Dr,


[email protected]

Robert Amor, Associate Professor,


[email protected]

ABSTRACT: Archaeological virtual environments are computerised simulations allowing the study and exploration

of archaeological sites. For architecture students and researchers at the University of Auckland they provide

several advantages compared to traditional methods of study and exploration such as site visits, illustrations and

books. Advantages include that there is no physical travel required, greater amounts of information can be provided

in a more accessible manner than with maps or diagrams, and different representations of the site can be created,

e.g., before modifications and expansions. The sites that archaeological virtual environments represent can contain

many structures and thousands of artefacts distributed over a wide area. As a result users find it hard to get an

overview of the site or to focus on particular aspects. Furthermore data on these sites is often gathered over a long

period of time using different processes and media, which makes it difficult to present effectively to a student body.

In this project we present solutions to these problems tailored to the needs of different user groups such as

archaeologists, architects and architecture students. The requirements of different user groups were analysed and

Virtual Reality technologies were developed to facilitate the exploration of archaeological sites, in order to retrieve

information effectively and to gain new insight into the site and its inhabitants. These technologies are demonstrated

within a new reusable archaeological virtual environment framework, which is used to create virtual environments

for archaeological sites. The framework is built upon a game engine, resulting in a quicker development cycle and

more realistic rendering than would be feasible if it were developed from the ground up. In contrast to previous

applications our framework enables the integration of a wide variety of media. This dramatically facilitates content

creation, which is usually very time consuming, expensive, and requires skilled modellers and/or animators. Our

framework provides simple interfaces to create a 3D context (terrain and simple models) and then integrates more

easily obtainable representations such as images and movies for providing visual details.

The technologies designed and implemented included the integration of QuickTime VR into a game engine, which

allows a commonly-used medium for recording scenes to be used within a virtual environment. The two media

integrate well and, while not seamless, the new representation enables a focus-and-context style exploration of the

domain. We also present a data model for archaeological sites that supports a wide variety of information types

including multimedia. It is independent of the rendering engine used, allowing archaeological virtual environments

to be extended and upgraded more easily. Using these representations we investigated new metaphors for

navigating and interacting with archaeological virtual environments, including interactive maps, guided tours,

searching mechanisms, time-lines and time-of-day settings for controlling sunlight direction.

The techniques afford users a richer, more informative experience of archaeological virtual environments. They can

be adapted to a broad range of archaeological sites, even where data was gathered by multiple differing methods.

KEYWORDS: Virtual environments, novel interaction interfaces, game engines, archaeology, data representations

139


1. INTRODUCTION & BACKGROUND

Three-dimensional virtual environments have been investigated since the 1990s for use in the study of archaeology.

Their uses range from purely research-oriented tools to aid in the visualisation of archaeological data through to

explorable digital renditions of full archaeological sites for uses such as education and virtual tourism. Whichever

the application, virtual environments provide a number of advantages:

• They allow people to visit sites without having to physically travel to them.

• They improve the ability of the observer to visualise and understand a site compared with traditional

media such as photographs and maps.

• They provide a more interesting and immersive view to the casual observer than such traditional

media.

• They allow a wider range of viewpoints from which to study a site, including in a temporal sense; for

example, an observer can watch a site develop over time

• They can present additional information on features of interest in a coherent, easily accessible manner;

for instance, an observer may examine details of an archaeological artefact presented in the virtual

environment without having to refer to a separate text. This is especially useful if the description

refers to the object’s location and its relationship the environment and other objects.

• They allow users to interact with the environment, e.g. by moving, adding and removing objects and

structures or by changing scene properties such as lights and materials. This can help understanding

the motivations for an existing architectural design and its changes over time.

• They allow people in separate geographical locations to work collaboratively on a single site.

• They can easily be distributed to the general public.

Archaeological virtual environments may be roughly divided into two classes; research aids and historical

reconstructions. The former represent the site as it exists today and provide the means to explore and study it; the

latter attempt to recreate the site as it was at one or more points in history. Both types of environments exist,

although historical reconstructions have been more widely publicised.

Virtual environments can be greatly beneficial for archaeological research. Before the advent of computer

technology, visualisations of archaeological data were created in the form of two-dimensional diagrams or

illustrations (Reilly and Rahtz 1992). The immense quantities of data generated from archaeological excavations and

surveys are often very difficult to adequately display in these formats, however. For example, stratigraphy – the

representation of the layers of earth on a site and the objects and soil types contained within – is represented using

two-dimensional diagrams called “sections” (Renfrew and Bahn 2004). This data is inherently three-dimensional

(Barceló et al 2000), and would be better presented using three-dimensional visualisation methods. Two-

dimensional methods also struggle to adequately represent the associations between finds on a site because of the

limited information that can be displayed (Barceló et al 2000). Interactive three-dimensional visualisations such as

virtual environments greatly ease this restriction, and provide an increased number of ways to visualise and work

with data. For example, neighbouring objects can be computationally analysed to determine whether they might

actually be fragments of a single object. Examples of these kinds of archaeological virtual environments include

ARCHAVE (Acevedo et al 2001), which allows the analysis of stratigraphy and the finds within an archaeological

site, and GeoSCAPE (Lee et al 2001), which facilitates the recording of the position of the features of an

archaeological site including artefacts, structures and other finds, and their later visualisation.

Virtual environments depicting historical reconstructions have often been aimed towards the general public, but they

can still be helpful to archaeologists. They may, for instance, be used to formulate and evaluate hypotheses about

questions such as how the site was used. They also allow a broader variety of ways to present information more

richly than earlier technologies: any number of viewpoints is possible in a virtual environment, features of the site

such as buildings and artefacts may be “hyperlinked” to relevant information such as bibliographic resources, and

querying can be used to locate and concentrate on important details (Renfrew et al 1996 and Barceló et al 2000).

One of the strongest advantages of these kinds of virtual environments is their usefulness for the dissemination of

research: archaeological research is normally published in a printed journal, and all too often a long time passes

140


between the actual fieldwork and analysis and its publication (Barceló et al 2000 and Renfrew and Bahn 2004).

Virtual environments can be distributed via electronic means such as the Internet. Furthermore, they can easily be

updated as new information emerges, increasing the frequency of the publication of this information as well as

reducing the delay. Virtual environments have the additional advantage of being easier to interpret than a series of

disparate diagrams and explanatory text, improving their accessibility to both academics and laypeople.

The use of virtual environments for “virtual museums” or tours aimed at the general public is also very common

(e.g. 3D Rewind Rome (2009), and the work of Gaitatzes et al (2001) and Kim et al (2006)). These are gradually

advancing in complexity and capabilities, with continuously improving graphics and some featuring sophisticated

animations as well.

Significant challenges are still faced in the development of virtual environments to represent archaeological sites,

especially as the scope and detail of the information presented increases. It is necessary to capture and digitize 3D

data before it can be used in a virtual environment, and it is difficult to achieve this cost-effectively while

maintaining high visual fidelity. Furthermore, information-rich virtual environments must merge data collected from

often disparate sources such as photographs, 3D models, GIS data and reference texts into a cohesive whole. This

data can be incomplete, tends to contain uncertainties, and may change over time as more information is collected.

The large quantity and complexity of the data available for an archaeological site necessitates careful consideration

of the representation of that data and how it is accessed. Our project has investigated methods to allow

archaeological virtual environments to present this data in a coherent, informative and easy to use manner, while

remaining cost-effective to create and maintain.

2. SELINUS CASE STUDY

During the development of our project we created a virtual environment of the ancient city of Selinus – a Greek

colony in Sicily – as a case study. The University of Auckland’s School of Architecture and Planning, in

conjunction with its counterpart in the Università degli Studi di Palermo (University of Palermo), runs a joint

program every few years in which students from Auckland and Palermo visit archaeological sites including Selinus,

and work together on architectural projects (Milojevic 2007).

FIG. 1: Overview of the Selinus site.

A multimedia DVD containing photographs, maps, 3D models, QuickTime VR panoramas, and textual resources

has been created to give these students an introduction to the Selinus site, its contents, and its history. However, it is

not as effective or as immersive as a virtual environment would be, and thus its authors wished to create such an

environment to improve the quality of their learning resources. We have used this project as an example to help

guide our decisions in the design of a virtual environment creation system that can be used for a broad variety of

purposes, which may not even include archaeological research.

141


3. REQUIREMENTS

In order to design a broadly-applicable virtual environment system we needed to consider the needs of the wide

variety of users it would have. These included:

• creators of virtual environments, such as researchers and other academics from the archaeological and

architectural fields;

• users from the architectural field who would be using the system to explore archaeological sites; and

• users from the archaeological field who would be using the system to study an archaeological site.

Users wanting to create virtual environments of a real location may have varying levels of computer skills, and are

often not programmers. Content creation tools such as level editors of game engines and modelling and animation

applications have made this task easier, but do require some artistic skills, reasonable technology skills (e.g. use of

scripting languages), and are time consuming and expensive. Most large-scale Virtual Reality application we are

aware of use professional designers and animators. A suitable application for laymen is “Second Life”, which,

however, does not have any tools for creating archaeological content. In order to make content creation more

efficient and effective we must utilise pre-existing data such as diagrams, 3D models, contour maps, and the like.

We surveyed users of archaeological environments and evaluated the literature and found (Keymer 2009) that users

of virtual environments from the architectural field tend to be most interested in the aspects of a site that could be

considered more “artistic”; its spatial features, its form, its appearance and so on. They are particularly interested in

the structures on a site, and thus are often responsible for creating historical reconstructions or visualisations of an

archaeological site or ancient building. In order to appreciate the form and shape of a site, they prefer to be able to

view them from as many vantage points as possible. Views from the ground are important, as they reflect the site

from the perspective a common person may have seen it; however views from the air can also be particularly

informative. Human activity is important to architects; this is one of the main reasons that the archaeological

artefacts on a site are interesting to them. The most important aspects of these artefacts are, again, largely artistic;

their minor details are less important than their presence, spatial attributes such as their arrangement, and what they

indicate about the human presence in the area. Thus architects may tend to browse through areas rather than study

particular groups of artefacts in detail.

Users from the archaeological field, by contrast, study sites from an anthropological perspective; they seek to

understand the people that lived there, not just the artefacts, buildings and other features that make up the site

(Renfrew and Bahn 2004). For this reason the crucial information from an archaeologist’s perspective is the cultural

significance of an artefact or a structure – for example who used it, how they used it, and what it indicates about the

society that created it. How this information should be presented depends on whether the virtual environment is

being used for presenting past findings or for original research; if the former is true an interpretation of the

significance of an object can be provided directly, but if the latter is the case as much information as possible should

be presented about each feature of the archaeological site to maximise the chances that a researcher will be able to

draw useful conclusions (Renfrew et al 1996).

These requirements can often conflict with each other; an environment that is useful for an archaeologist is likely to

contain a great deal of information which would be unnecessary or even distracting to an architect. Therefore a

virtual environment creation system that is useful to both would need to be able to tailor the resulting virtual

environment to present the level of information required.

4. PROJECT DESIGN - GAME ENGINE

The rendering framework was the first stage of the development of the project as it would form the foundation for

later work. We elected to build it using a pre-existing framework or technology as a base. There were three main

reasons for this. Firstly, the development of a new framework would not address any of the goals of this project; it

focuses on user interface techniques, and features such as a custom rendering engine were considered to be unlikely

to add significant value to these. Secondly, the use of a pre-existing framework would accelerate the development of

the system overall, and the features that would add value in particular. Thirdly, the disadvantages and limitations of

such an approach are minor; although it may limit the flexibility of the overall system somewhat, this can be largely

mitigated with careful choice of a technology appropriate to the project.

142


We analysed a variety of options when choosing a basis for our virtual environment rendering framework, including

general-purpose 3D visualisation technologies, previously developed archaeological tools, and game engines

(Keymer 2009). Game engines fulfilled our requirements best since they are cheap, are optimised for consumer level

hardware, are frequently updated to use the latest graphics technologies, provide intuitive interaction tools, and have

a large user base resulting in well tested and stable code. We eventually settled on Esperient Creator (2009), as we

received access to the SDK, had previous experience with it, and because it is highly flexible, had excellent tools,

strong plug-in support, multi-player support (for collaborative visualizations), and is targeted at the development of

non-game virtual environments such as those we wish to support.

Even with a well-suited engine, however, we still needed to create our own pipeline for creating, rendering and

exploring archaeological virtual environments. We achieved this through the use of two major components: one of

these is a stand-alone program to collect various types of archaeological data, compiling them into a computerised

description of the archaeological site. The other component is a plug-in for the Esperient Creator engine which

interprets this description and uses it to display and facilitate interaction with the virtual archaeological site.

This plug-in is designed to be re-usable and extensible, in order to support a wide variety of kinds of archaeological

sites. It has a two-layer, modular architecture which separates the concerns of the data model from the engine used

to represent it – making the data easier to work with and more portable – and separates distinct blocks of

functionality into their own modules, which helps to make new functionality easier to introduce.

FIG. 2: Virtual environment creation interface: terrain loading and modification.

4.1 Data Model

Archaeological data sets can be extremely large, diverse and feature rich. In addition some data, such as building

structures, can change over time. We have created a custom data model which supports dynamic scene generation,

efficient data access and manipulation, and interaction with other systems or APIs. The data is imported from an

XML scene descriptor file, which facilitates interchange of data, extension of the model, and integration with web

interfaces. The data model is divided into the following components which have different attributes and functions:

• Environmental components (terrain, seas, rivers, vegetation)

143


• Man-made structures (buildings and other structures such as walls and moats)

• Archaeological artefacts

• Multimedia components (e.g. photographic images of the site and objects, including QuickTime VR

panoramas and object movies.)

• Temporal entities (eras)

All of these components can be interrelated. For example, man-made structures and artefacts can be associated with

one or several eras and multimedia components can be associated with positions in the terrain or artefacts. Each of

these data entities contains attributes describing it, such as size, shape, age, date created, date found, bibliographic

references, multimedia descriptions (photos, object movies tec.), and relationships with other elements of the

archaeological site. More details of the data model and attributes are found in (Keymer 2009).

Different data entities are generated and loaded using different tools using common interfaces. For example, figure 2

illustrates the terrain generation using a digital elevation map (DEM) representation where height values correspond

to gray scale values of an image. We have also developed an application for creating DEMs by drawing and semi-

automatically labelling contour lines or importing them from GIS applications (Xie and Wünsche 2009).

FIG. 3: QuickTime VR within the game environment.

4.2 QuickTime VR support

Supporting some types of data used to create archaeological sites can be more difficult than others; one of the most

difficult was QuickTime VR panoramas. These allow a user to view a 3D scene from a single point by wrapping a

144


photograph or series of photographs onto a virtual cylinder or sphere. The user can click and drag with the mouse to

rotate their view of the scene. QuickTime VR panoramas are widely used by a number of archaeological projects,

including the Selinus project, because they can be produced relatively easily and inexpensively, and allow an

immersive view of a site that is otherwise difficult to obtain.

We allow the user to transition from a view of the virtual environment to a view of the QuickTime VR panorama,

retaining the same control system from the environment, which is also drawn around the border of the QuickTime

VR panorama to provide a frame of reference. The QuickTime VR panorama provides an interactive feature rich

representation without expensive and time-consuming creation of 3D content. Note that 3D content can be created

automatically from video images, but this is computationally expensive, heavily influenced by environmental

parameters (illumination, shadows), and requires expensive well calibrated hardware. In addition the resulting

scenes usually contain many small errors which severely reduce viewing pleasure.

This process was challenging to implement; QuickTime is normally designed to be used by embedding a player in

an appropriate place inside an application window, but due to issues in timing the rendering of the player with the

in-built rendering cycle of Esperient Creator, we had to introduce an extra stage of indirection into the process by

rendering the player to a non-visible section of memory, the contents of which were then copied into an Esperient

texture buffer. Details are given in (Keymer 2009). While this process is more complicated and less efficient than

drawing directly to the screen, it does allow us more flexibility with the use of the QuickTime player's output. The

results are illustrated in figure 3. The view direction inside the virtual environments changes with the view of the

QuickTime VR movie such they are always aligned. Small offsets are unavoidable due to inaccuracies in the

modelled scene and lack of information about the exact camera position for the QuickTime VR movie.

4.3 Navigation

Another important concern is how users move through and navigate the virtual environment. Architects – as we

discovered – are particularly interested in being able to view a site from the ground, but also find it helpful to view it

from the air. Thus our system had to handle both, allowing the user to switch between them quickly and easily.

Ground-based movement had the higher priority of the two.

The size of large sites – such as Selinus – also has impacts on navigation. It will take some time to cross a large site

if the user can only move slowly; however if movement is too fast the system will be less usable and less immersive,

and disrupt the user’s perception of the scale of the site. The system had to allow the user to move at a normal

walking speed, but also provided methods for crossing large distances that do not disrupt perception of scale or

immersion. The portal concept from computer games is a suitable method to jump quickly between logically

connected positions, e.g. buildings which related functions spread over a wide area. The Strider concept, which we

introduced in (van der Linden et al. 2004), allows exploration with smoothly varying speed, context and perspective.

A major difference to games and other virtual environments is that archaeological sites change over time. The user

should hence not only be able to explore a site in the spatial domain, but also in the temporal domain. The time

period displayed in the virtual environment is controlled by a “timeline” slider as illustrated in figure 4. The slider is

manipulated using the mouse, and the adjacent text provides a visual indication of the currently selected era. Only

objects valid during that era are rendered. In order to achieve this each data model entity – presently artefacts and

buildings – that is affected by the timeline system contains a list of the eras it is present in and absent from. Terrain

is less likely to change over time, but the system could be extended to support terrain if such changes are later found

to have been significant.

Guided navigation methods – such as tours – allow users to become acquainted with a site quickly. While non-

essential we believe they would be a valuable addition to our system. Tours should utilise both the spatial and

temporal domain and should be easy to find, easy to use, and effective at improving understanding of the site.

The system should also support a mechanism that allows artefacts and other important elements of an archaeological

site to be found quickly. This would be especially important for archaeologists – to allow them to locate related

finds, for instance – but it would also be helpful for anyone who is particularly interested in one location or artefact

on the site. This kind of “search” mechanism would therefore need to be designed to identify individual entities

based on a variety of commonly-required constraints, whether by individual object attributes, groups of attributes, or

relationships to other objects on the site.

145


FIG. 4: Timeline control to filter site view.

4.4 Artefact display

There are a number of key requirements the artefact handling features of the virtual environment system had to

satisfy. These include abilities such as being able to represent the artefacts on screen, conveying as much

information about them – especially important details – as quickly and concisely (such as in visual form) as possible,

allowing the user to easily interact with them as necessary, and managing situations where artefacts are distributed

quite densely.

We chose to represented artefacts within the virtual environment by small 3D objects, or “icons”, which indicated

the type of the artefact; for instance an individual amphora may be represented by a generic “amphora” icon.

Because there are relatively few artefact types, only a few models need to be provided, which reduces the amount of

time and resources necessary to build virtual environments containing large numbers of objects.

When the icon is clicked a “detail view” is shown which displays more information about the selected artefact. This

view displays material that cannot be easily displayed in the virtual environment – such as textual information and

visual media – and information that may be represented in the environment but cannot be accurately interpreted

there – such as its dimensions and significance.

The detail view consists of two “panes” as in figure 5. The left pane displays a selection of visual media, while the

right contains exclusively textual information about the artefact. Media in the left pane need not be static images;

QuickTime VR object movies are shown here as well, and the system is constructed to be extended to support other

media types including animations and movies as needed. This is demonstrated by the QuickTime VR integration

which is essentially an advanced movie format.

5. RESULTS, CONCLUSIONS AND FUTURE WORK

We have presented a system supporting the display and exploration of archaeological virtual environments that can

be created from a broad range of types of media. This work was motivated by our observations that much existing

data recorded for archaeological sites is difficult to use in existing virtual environments, and that they could be

improved if these types of data were supported. After analysing the problem and user requirements we identified key

features to incorporate into our system, then developed designs for these features and implemented them into a

working application.

146


We have evaluated technologies and found that game engines are best suited because using they provide multi-

player capabilities, intuitive interfaces, tools for content creation, and they support common 3D data models. Our

system separates the content of a virtual environment from the system used for the presentation of that environment.

This helps it to achieve the objective of supporting a wide range of archaeological sites. We created a conceptual

“meta-model” of an archaeological site describing the kinds of features and information that can make up an

archaeological site; this information is used to create collections of data files that can be used by the same

environment. One of the most effective features of this approach is that it allows pre-existing data that has been

collected for an archaeological site to be used for creating a virtual environment. This data may be stored in a

diverse range of formats, such as GIS data, photographs, models and QuickTime VR movies. By using this data as-

is rather than requiring custom-designed content be used for the site, we have created a system that allows virtual

environments to be created more easily, quickly and efficiently. Furthermore it can be updated simply by adding

new content, without requiring an expensive and time-consuming conversion into a 3D representation.

FIG. 5: Object icons in the game environment and the detail view.

Integrating QuickTime VR movies into the rendering of the scene provided rich content with suitable context.

Identifying the precise camera position for a QuickTime VR view is difficult without additional data such as its GPS

position. However, even without seamless integration the media blended well and allows a focus-and-context style

exploration of the domain. The game environment provides an overall context for QuickTime VR movies and

enables users to get an overview of the entire scene and the QuickTime VR movie’s camera position within it. Vice

versa the QuickTime VR movie is content rich and provides a more realistic representation than the modelled

environment and hence provides a better context for understanding the terrain, e.g. how the architecture blends into

the natural environment.

We also performed a basic usability study and found that navigation is easy, although the existing Selinus virtual

environment is relatively simple. More challenging scenarios are required to test how well the system performs in

complex environments. Interaction with QuickTime VR panoramas works quite well. They are easy to see and

147


activate, and view rotation is synchronised accurately. Rotating the view with the mouse results in sluggish

handling, however, which needs to be improved. Interaction with artefacts is relatively easy, but they can be difficult

to see at a great distance due to their size. The planned inclusion of a top-down map should alleviate this problem.

The Selinus site takes a long time to move around at present, and this could become frustrating for users. This may

be partly due to performance issues that slow down movement, but in any case it is likely that faster movement, e.g.

by using the previously mentioned Strider interface, will be required.

Overall our results indicate that the developed tool is intuitive to use and provides an easy way to create content rich

multi-media supported 3D representations. More detailed user studies are required to test the effectiveness of the

tool. In particular we want to test whether our tool helps users to explore an environment more efficiently, whether

users’ memory of scene content and spatial relationships improves, and whether users can solve archaeological

problems more effectively. We also want to do add more capabilities for automating the creation of scene content,

e.g. modelling of 3D objects and structures from images using photometric stereo, integrating GIS data (if

available), and automatically creating guided tours based on spatial, temporal and semantic relationships.

6. REFERENCES

3D Rewind Rome (2009), [cited 25 January 2009]; Available from: http://www.3drewind.com/.

Acevedo, D., Vote, E., Laidlaw, D.H. and Joukowsky, M.S. (2001), “Archaeological data visualization in VR:

analysis of lamp finds at the great temple of petra, a case study”, Proceedings of Visualization '01, IEEE

Computer Society, San Diego, California, 493-496.

Barceló, J.A., Forte, M. and Sanders, D.H., eds. (2000), “Virtual reality in archaeology”, BAR international series,

Vol. 843, Oxford, England, 262pp.

Esperient Creator (2009), [cited 15 July 2009]; Available from: http://www.esperient.com/.

Gaitatzes, A., Christopoulos, D. and Roussou, M. (2001), “Reviving the past: cultural heritage meets virtual reality”,

Proceedings of Virtual reality, archaeology, and cultural heritage, ACM, Glyfada, Greece, 103-110.

Keymer, D.J. (2009), “User Interfaces for the Effective Exploration and Presentation of Virtual Archaeological

Sites”, ME Thesis, University of Auckland, Auckland, New Zealand.

Kim, Y.-S., Kesavadas, T. and Paley, S.M. (2006), “The Virtual Site Museum: A Multi-Purpose, Authoritative, and

Functional Virtual Heritage Resource”, Presence: Teleoperators & Virtual Environments, 15(3), 245-261.

Lee, J., Ishii, H., Dunn, B., Su, V. and Ren, S. (2001), “GeoSCAPE: designing a reconstructive tool for field

archaeological excavation”, CHI '01 extended abstracts on Human factors in computing systems, ACM,

Seattle, Washington.

Milojevic, M. (2007), “Exceptional Access: Re-Presenting Ancient Selinus Virtually”, Proceedings of Interface:

Virtual Environments in Art, Design and Education, Dublin, Ireland, 6-7 September.

Reilly, P. and Rahtz, S., eds. (1992), “Archaeology and the Information Age: A Global Perspective”, Routledge,

London, England.

Renfrew, C., Forte, M. and Siliotti, A., eds. (1996), “Virtual archaeology: great discoveries brought to life through

virtual reality”, Thames and Hudson, London, 294pp.

Renfrew, C. and Bahn, P.G. (2004), “Archaeology : theories, methods and practice”, Thames & Hudson, 4th ed.,

London, 656pp.

van der Linden, J., Li, J., Lobb, R., Novins, K. and Wünsche, B. (2004), “Strider: A Simple and Effective Terrain

Navigation Controller”, Proceedings of IVCNZ '04, Akaroa, New Zealand, 21-23 November, 95-100; URL:

http://www.cs.auckland.ac.nz/~burkhard/Publications/IVCNZ04_vanderLindenEtAl.pdf

Xie, X. and Wünsche, B. (2009), “Efficient Contour Line Labelling for Terrain Modelling” [submitted for

publication]

148


INTERACTIVE CONSTRUCTION DOCUMENTATION

Antony. Pelosi, Mr,

Massey University;

[email protected]

ABSTRACT: This paper discuses the potential of moving beyond 2D paper-based construction documentation by

leveraging the power of real-time 3D computer gaming engines to produce first person view digital environments to

explain design intent for the architectural, engineering and construction industries. By comparing methods used in

the aerospace and automobile industries to requirements of the AEC industries, this paper outlines the benefits of

interactive 3D construction documentation.

KEYWORDS: Architectural hyper-model, computer game engine, real-time 3D, interactive, construction

documentation.

1. INTRODUCTION

There are inefficiencies in the production and editing of working drawings. Building information modeling (BIM)

has started to reduce these inefficiencies, but it remains focused on 2D paper based drawings as the final output,

subsequently losing the benefits of an intelligent 3D model. This paper will examine how to leverage BIM and 3D

computer gaming engine technology to form an architectural hyper-model that would be a valuable supplement to

the conventional scaled 2D construction drawing documentation found on construction sites. Professor Chuck

Eastman et-al states that, “Three-dimensional models and 4D simulations produced from building information

models are far more communicative and informative to lay people than technical drawings” (Eastman et al., 2008).

With the increasing availability of powerful real-time 3D digital environments, like Google Earth and computer

games, the next generation of people in the building industry are already comfortable with interacting and

communicating within real-time 3D digital environments. The Construction 2020 report prepared by the

Cooperative Research Centre (CRC) for Construction Innovation outlines visions and suggests goals for the

construction industry. They state “…for communication and data transfer to be seamless, enabling transfer without

interruption and delay, and include mobile devices providing a commercially secure environment. These

technologies will be embedded within both construction products and processes to improve efficiency and

effectiveness. The knowledge economy will require property and construction to become more engaged in IT

developments.”

2. CONSTRUCTION VISUALIZATION

Historically, architects have produced a set of 2D drawings and specifications that have abstracted information

spread across sheets of paper at different scales and points of view. Coupled with the fact that a set of drawings is

typically multiple sheets, the proposed building project is made difficult to visualize as a complete object as well as

the narrative they describe for the process of construction. Contractors have to piece together information from 2D

drawings; this form of communicating has been the established practice for over 500 years. The methods of

architectural design visualization have undergone major changes over the past 50 years with the introduction of the

computer and CAD software in to architectural practice, but we have seen very little improvement or innovation to

construction documentation over the same period.

The current shift in the AEC industries is to building information modelling (BIM). Professor Chuck Eastman et al,

define BIM in their book BIM Handbook as “a modelling technology and associated set of processes to produce,

communicate, and analyze building models.

• Building components that are represented with intelligent digital representations (objects) that 'know’

what they are, and can be associated with computable graphic and data attributes and parametric rules.

• Components that include data that describe how they behave, as needed for analyses and work

processes, e.g., takeoff, specification, and energy analysis.

149


• Consistent and non-redundant data such that changes to component data are represented in all views of

the component.

• Coordinated data such that all views of a model are represented in a coordinated way.

They then go on to quote Mortenson Company, a construction contracting firm’s definition of BIM technology as,

“an intelligent simulation of architecture.” To enable Mortenson to achieve integrated delivery, this simulation must

exhibit six key characteristics:

• Digital;

• Spatial (3D);

• Measurable (quantifiable, dimension-able, and query-able);

• Comprehensive (encapsulating and communicating design intent, building performance, constructability,

and include sequential and financial aspects of means and methods);

• Accessible (to the entire AEC/ owner team through an interoperable and intuitive interface); and

• Durable (usable through all phases of a facility’s life).

The use of BIM in practice has many well-documented benefits for the AEC industries (Eastman et al., 2008, Issa

and Suermann, 2007) The majority of these benefits focus on the design and pre-construction phase of building and

not the transfer of information to parties new to a project or who are only required to deal with a small element, sub-

contractors for example.

Professor S. N. Pollalis from Harvard Design School comments that “Computers today are used to produce 2D

drawings faster and, as a result, in large quantities. However, one should question the basics and re-deploy

computers to solve the problem of over-documentation as opposed to using them for multiplying the existing spill-

out of information” in a paper tilted Understanding Changes in Architectural Practice, Documentation Processes,

Professional Relationships, and Risk Management. (2006)

Research from Center for Integrated Facility Engineering (CIFE) at Stanford and the National Institute of Standards

and Technology (NIST) show how traditional practices are inefficient and error prone, contributing to budget

overruns. The study by CIFE illustrated that productivity in the construction industry has remained constant over the

last 40 years while just about every other industry has seen productivity growth.

The results from a study carried out by NIST show that “inefficient interoperability accounted for an increase in

construction costs of over US$6 per square foot for new construction and an increase of US$0.23 per square foot for

operations and maintenance, resulting in a total added cost of US$15.8 billion [in America].” (Gallaher et al., 2004)

In the context of the NIST study, one of the key characteristics of inadequate interoperability is mitigation of digital

and/or paper files that had to be manually re-entered into multiple systems and request for information management.

One of the benefits of the data re-entry is the increased spatial understanding of the proposed building for the people

entering the data, however this knowledge is not past on. Considering this fact, how can this increased spatial

understanding be delivered H without costly data re-entry and be accessible to all parties involved in the life of a

building? After evaluating methods used in other industries and comparing the different requirements, this paper

proposes that the answer lies with the use of real-time 3D interactive digital environments.

The use and accompanying benefits of 3D models and BIM have largely been limited to design efficiencies, conflict

detection, quality improvements and on time completion (Issa and Suermann, 2007), with the final output still

emulating 2D hand drawn documents. BIM technology is currently focused on the creation of the model, not the

accessibility of the model, only providing an increased number and complexity of drawings and specifications to the

construction contract documents. (Pollalis, 2006).

The role of paper-based 2D working drawings is set to change over the next ten years. It is still unclear what these

changes will be and the impact on the AEC industries. BIM will be influencing these transformations. One of the

roles of drawings in the construction industry is as contractual documents; professor Chuck Eastman et al state,

“there are indications that BIM models can better serve this purpose, partly because of their improved accessibility

to non-professionals” (2008). The question remains how do you communicate design and construction intent to the

people building a building? Many architects already provide axonometric and perspective drawings as part of the

150


drawing set to help explain design intent, with positive feedback from contractors. Current drawing standards have

evolved due to the technological limitations of paper and the tools used to draw; orthographic projects were

developed to enable measuring of distances on paper. If we are to move beyond a paper based delivery system, then

the current drawing symbols and formatting conventions will need to be replaced to reflect the new methods.

3. DIGITAL DOCUMENTATION PROCESSES

This paper reviews digital documentation processes used in the aerospace and automobile industries, gaining an

understanding of their workflows and outputs in relation to the construction industry; the new Boeing 787

Dreamliner has completely paperless documentation, for example.

The aerospace and automobile industries have been leading the implementation of CAD, solid modelling, product

life management (PLM) and digital documentation, providing huge productivity gains across the entire life of

production, from design through to maintenance. The study by NIST – mentioned in section 2 – notes that

“Computer, automobile, and aircraft manufacturers have taken the lead in improving the integration of design and

manufacturing, harnessing automation technology, and using electronic standards to replace paper for many types of

documents. Unfortunately, the construction industry has not yet used information technologies as effectively to

integrate its design, construction, and operational processes. There is still widespread use of paper as a medium to

capture and exchange information and data among project participants”. (Gallaher et al., 2004)

There has been a shift from 2D line based graphic documentation to real-time 3D interactive models for manufacture

and maintenance within the aerospace and automotive industries. They are developed directly from the design CAD

3D model, resulting in reduced production times for 3D and 4D graphic documentation and huge cost savings with

faster time to market. Bell Helicopters have updated their graphical documentation processes, with the technology

shift saving 39,000 labor hours in development of technical publications for one aircraft alone. “That’s an 80-

percent savings in the time to produce graphics of much higher quality, and that are much more portable,” resulting

in netted savings in another project of 26,000 labor hours a year.

Studies have shown that people are more likely to retain information by ‘doing’ versus only reading (Dale, 1969).

Interactive 2D, 3D and 4D training contents simulate ‘doing’. Virtual simulations, and virtual reality are a lower cost

alternative to actual product mock-ups, physical simulations, and other non-electronic ‘learning by doing’

approaches. A white paper published by visual product communication and collaboration company, Right

Hemisphere, states that “Interactive training has become mission critical in aerospace, defence, automotive, and

other industries” (Right Hemisphere, 2005)

To achieve fully digital documentation companies like Boeing, Bell Helicopters and Daimler Chrysler are using

programs like Right Hemisphere’s Deep Exploration with Adobe 3D pdf’s and real-time 3D authoring programs

such as Esperient Creator to leverage complex CAD data for manufacture and maintenance training documentation,

thus providing non-paper based communication of complex procedures that have historically been expensive and

complex to produce and distribute globally. The digital multidimensional documentation has reduced time to market

and costs by providing a platform that offers a secure, clear image based multi-lingual instruction package. This is

all achieved with great interoperability, without the need to re-input the core design information, reducing expensive

authoring time once design is complete.

The automotive and aerospace industries have large lead times for design with hundreds or thousands of human

hours working on one project that will be mass-produced in the hundreds and thousands or even millions, for global

distribution and consumption. The resulting design CAD model is an exact dimensional correct representation of the

finished product, compared to a building project that is usually a single output of a design solution dealing with

localized climatic conditions, building code requirements, and different client demands and expectations. The design

CAD model has a dimensional different condition to the building, usually based on a grid system that controls the

areas of tolerance.

CAD packages developed for the automotive and aerospace industries have been used in the AEC industries with

outputs that would not have been possible with standard AEC software. Frank O, Gehry’s Guggenheim Museum,

Bilbao is one of the most notable examples. Dassault Systems’ CATIA, the world's most widely used parametric

modelling platform for large systems in aerospace and automotive industries (Eastman et al., 2008) is the backbone

for Gehry Technologies Digital Project, BIM software tailored to the building industry.

151


There has been very little change in the type of documentation of design intent in the AEC industries since the

introduction of drawing software into the workflow. Mentioned in section 2 there has been an increase in

documentation due to a change in relationship between architect and contractor, and a raise in litigation risk.

4. ARCHITECTURAL HYPER-MODEL

How can the AEC industry learn from the techniques that are being used in the aerospace and automobile industries?

What elements or methods will work in the AEC industries and what information needs/requirements can be

provided with more clarity than current methods?

This paper proposes that as the AEC industries move towards BIM and as the work force becomes dominated with

people who have grown up with Playstaion and Xbox 3D video games, we rethink methods of providing and

viewing complex building design intent that is not based on a 2D paper paradigm, but supplemented by. The cost of

printing and distributing paper documentation adds up to millions of dollars in lost revenue every year—not to

mention the ecological impact—within the AEC industries. Delays involved in physically distributing designs, the

difficulty of keeping distributed workers up to date, and the inability to track feedback on paper often result in costly

mistakes. Secondary and directly measurable costs for creating and delivering printed designs also remain.

(Autodesk, 2008)

The AEC have established digital file formats and systems to speed up a paper-based workflow that are independent

of the original authoring application for representing design data. Such as Design Web Format (DWF) developed in

the mid nineties by Autodesk for the efficient distribution and communication of rich design data, DWF was created

as a 2D only file format and has evolved to incorporate 3D data. The software developed to view the DWF files tries

to offer a Swiss army knife approach to accessing the model, providing every possible tool set to view, review and

mark-up, or print design information, resulting in confusing interface and complex learning curves. This technology

is based on a 2D paper based paradigm, resulting in problems of comprehension of printed 2D drawings.

The author wrote in a previous paper that “Currently most CAD programs work within an ‘object’ based mode,

using legacy methods and concepts from the drawing board. An architect or designer constructs a building or object

in digital space and then zooms in and out, rotating and panning around it. This type of navigation loses any sense of

scale, context and relationships between items. Gravity and sense of ground are also eliminated or suspended. These

representation modes impede comprehension of the building or object. Computer gaming software uses an

‘environment’ mode rather than an object mode to display architectural space, giving a viewer centric point or first

person perspective view. The shift from object based modelling to environment based modelling changes the way

the information can be viewed and understood, providing a scaled space which is navigated by walking or flying and

hyper-linking in real time. The first person view gives a stronger understanding of scale and relationships of the

proposed building. The current technology enables these spaces to be dynamic, with items which can be moveable,

even picked up and changed, using real world physics.”

Ongoing research at a number of universities and companies around the world into the use of computer game-engine

technology in the AEC industries points to a possible direction to head. Autodesk labs, a new technologies

innovation centre, have announced a new technology tilted ‘Project Newport’ – “real-time 3D story building

technology for architectural visualization and presentation. With game-engine technology and breakthrough ease-of-

use, Project Newport enables architects to show their design in context, rapidly explore design options, and create

vivid and immersive 3D presentations. Newport brings architectural designs to life by expressing design intent at

every stage of a project.” A number of other industries are using and researching the use of computer game-engine

technology for information visualisations.

Research to date has started to show that an architectural hyper-model could be used to better understand

construction documentation to improve productivity. It has become clear the possibilities of this technology include:

• A scalable view: the content can be viewed on any sized screen with little or no modification (projected

images, desktop and hand-held screens). Current 2D paper-based documentation is formatted for a single

size sheet that requires completely reformatting to be viewed on a hand-held computer or other platform.

• Easy navigation control: based on standard computer gaming navigation conventions, in 2008, figures from

The Entertainment Software association of America showed that 65% of American households play

152


computer or video games, with the average game player age being 35. Current CAD software can be

confusing to navigate around due to its object based navigation focus.

• Specification integration and material identification: with the use of realistic and coded textures that can be

directly linked to procurement and specification information. Creating hyper links between drawings and

specifications enable easy access between the model and non-model information or meta-data.

• Fully Interrogative: unlike pre-rendered animated walk-throughs, the direction, the speed, and the sequence

of movement are not forcefully prescribed or scripted. This effectively eliminates the passive nature of the

animated walk-through and allows to the participants complete and thorough interrogation of the rendered

spaces. (Hoon et al., 2003)

• Real-time physics: the use of a physics engine enables to the user real world physical properties within the

model providing to the viewer object collision with walls and floors, and the ability to have user controlled

moveable objects within the 3D environment.

• Construction sequence visualization: “Properly implemented 3D and 4D CAD systems can improve

communication of the construction process and therefore close the proverbial gap between the idea and the

implementation.”(Saha and Hardie, 2004)

• Integrated details: hyper-linking from model to details in real-time connected to a project database

providing fast and correct connections to details and further information.

• Multiples: unlike physical drawings and models, a hyper-model can be viewed in multiple locations at once

(only requiring additional hardware).

• Collaboration: multiple simultaneous users allow real time virtual meetings within the hyper-model;

facilitating designers, contractors and owners in different physical locations to meet in 3D digital space and

discuss issues within the context of the model.

• Digital Document Management: the hyper-model would be stored online ensuring the latest model is

accessed by all and eliminating the potential for multiple versions, leading to miscommunication.

Due to the early stage of research of this project, with currently in true limitations, an architectural hyper-model on a

construction site have not be fully explored to date. It is expected that foreseen limitations can be managed through

design and formatting of interface per device or screen size requirements. More work is necessary to fully

understand these issues.

5. CONCLUSION

Advances in technology and the move towards building information modelling in the AEC industries are pointing to

a time to re-evaluate construction documentation methods. The processes used in the aerospace and automobile

industries demonstrate possible directions to begin the review. The maturing computer gaming industry is providing

powerful real-time 3D digital software that has potential to supplement current 2D paper-based construction

documentation. Providing an architectural hyper-model as part of the construction to improve understanding of the

design intent will improve construction efficiency. Future research will focus on developing methods and measures

to observe potential limitations and productivity gains of implementation of an architectural hyper-model into a BIM

workflow.

6. REFERENCES

Autodesk (2008). DWF: The best file format for published design information http://www.autodesk.com/dwf-whitepapers.

Dale, E. (1969). Audiovisual methods in teaching, New York,, Dryden Press.

Eastman, C., Teicholz, P., Sacks, R. & Liston, K. (2008). BIM handbook: a guide to building information modeling for owners, managers, designers, engineers and contractors, Wiley.

ESA (2008). Essential facts about the computer and video game industry. The Entertainment Software Association.

Gallaher, M. P., Oconnor, A. C., Dettbarn, J. L. & Gilday, L. T. (2004). Cost analysis of inadequate interoperability in the u.s. capital facilities industry

153


Gallo, G., Lucas, G., Mclennan, A. & Parminter, T. (2002). Project documentation quality and its impact on efficiency in the building and construction industry. Institution of Engineers Australia.

Gao, Z., Walters, R. C., Asce, M., Jaselskis, E. J. & Asce, A. M. (2006). Approaches to improving the quality of construction drawings from owner’s perspective. Journal of Construction Engineering and Management.

Hoon, M., Jabi, W. & Goldman, G. (2003). Immersion, interaction, and collaboration in architectural design using gaming engines. Proceedings of the 8th CAADRIA Conference. Indianapolis, IN, USA.

Issa, R. & Suermann, P. C. (2007). Evaluating the impact of building information modeling (BIM) on construction. 7 International Conference on Construction Applications of Virtual Reality:.

Kalay, Y. E. (2004). Architecture's new media : principles, theories, and methods of computer-aided design,

Cambridge, Mass. ; London :, MIT.

Pelosi, A. (2007). Architectural hyper-model: changing architectural construction documentation. Association of

Architecture Schools of Australasia. Sydney.

Pollalis, S. N. (2006). Understanding changes in architectural practice, documentation processes, professional relationships, and risk management. gsd.harvard.edu.

Right-Hemisphere (2005). product graphics management for interactive 3d training.

Saha, S. & Hardie, M. (2004). The use of 3d and 4d cad systems to reduce error rate and reworking in construction. Clients Driving Innovation Conference.

Thomas, M. (2004). A graphics pipeline for making 3D as cool as text. ACM SIGGRAPH Computer Graphics.

Tilley, P. A. (2005) .Design and documentation quality problems—a lean thinking opportunity. Proceedings of the

International SCRI Symposium.

154


CASE STUDIES ON THE GENERATION OF VIRTUAL ENVIRONMENTS OF REAL WORLD FACILITIES

Michele Fumarola, Ph.D. Candidate,




Tel. +31 (0)15 27 89567

[email protected]

http://www.tudelft.nl/mfumarola

Ronald Poelman, Ph.D. Candidate,




Tel. + 31 (0)15 27 88542

[email protected]

http://www.tudelft.nl/rpoelman

ABSTRACT: There is an increasing need to generate detailed real-time virtual environments that closely mimic real

world facilities. Drivers for this need are wider application of virtual training environments, new virtual pre-testing

of design, and joint virtual development of information systems. Approaches for generating virtual environments

vary from manual to automatic. In manual approaches we may have to go to the location, take pictures to develop

the 3D model and thereby construct the virtual facility. This approach is time consuming, inaccurate and coarse. In

automatic approaches we could use technology to capture the facility, e.g. laser-scanning, photogrammetric

methods and radar. Based on the data acquired a virtual environment can be generated automatically.

Unfortunately, automatically generated data sets are less than optimal for practical use within real-time virtual

environments because of the huge unstructured amount of data. Common approaches are therefore most likely to

have a balance between human and computer effort. Computers are typically suitable to do bulk work, where

humans are well suited to pick-up unnatural phenomena. Based on different case studies, we discuss the distribution

of manual and automatic methods for the generation of 3D virtual environments. Different facets of the pipeline

from initial data gathering up to a final deliverable are presented. The case studies vary from fully hand made up to

semi automatic reconstruction of the environments. The paper concludes with recommendations regarding the

reconstruction methods.

KEYWORDS: Virtual environment, virtual reconstruction, visualization, serious gaming

1. INTRODUCTION

1.1 A need for 3D virtual environments based on real world facilities

There is an increase in use of 3D virtual environments in architecture, engineering and construction (AEC)

industries. Applications like virtual training environments, virtual pre-testing of design, and joint virtual

development of information systems require a valid representation of the real environment. Often, the real

environments are industrial facilities such as oil rigs, containers terminals, or manufacturing plants.

There is no single reason why 3D virtual environments are increasingly popular, but drivers stem from multiple

backgrounds. From a theoretical perspective advantages of 3D virtual environments are found in improving

communication (Arayici and Aouad 2004), increasing insight (Shiratuddin and Thabet 2007; Woksepp and Olofsson

2006), supporting collaboration (Bouchlaghem et al. 2004), and supporting decision-making (Kam and Fischer

2004). The divergence in applications requires different levels of fidelities of the 3D virtual environment. This can

be illustrated by the different levels of fidelity required in the design process of a manufacturing plant. Designing the

plant is mostly done in a 3D environment with high precision. The design drawings are complex, show different

layers (e.g. mechanical, electrical, and plumbing) and therefore become hard to understand. On the contrary, for the

155


presentation of the final result to stakeholders, a 3D visualization with a reduced level of complexity is preferred.

The different types of visualizations achieve different types of fidelity for each specific goal.

1.2 Realism in 3D virtual environments

According to Webster’s dictionary, ‘fidelity’ means the accuracy in details. While fidelity is the general term for the

way in which a model is a valid representation of a reference system, 3D modellers tend to use the term ‘realism’, as

their reference system is the real world, i.e. the real industrial facility. Ferwerda (2003) distinguishes three varieties

of realism in computer graphics: physical realism, photo-realism, and functional realism. For each type of realism,

there is a criterion which needs to be met in order to achieve that type of realism.

Physical realism is achieved when computer graphics provide the same visual stimulation as reality. This type of

realism means that “the image has to be an accurate point-by-point representation of the spectral irradiance values at

a particular viewpoint in the scene”. It requires an accurate description of the scene, simulation of the spectral and

intensive properties of light energy and reproduction of those energies by the display device. Technically this type of

realism is the hardest one to achieve. While this aspect is often ignored for more interpretational and overview

models that are geared towards human observers, it might become essential in 3D virtual environments for future

use. In this paper physical realism is ignored.

Photo-realism in a virtual scene provides the same visual response as the real scene. It aims at displaying an image

indistinguishable from a photograph of the real scene. Although achieving photo-realism has primarily been a task

for off-line rendering algorithms, modern interactive 3D visualization software libraries tend to surpass the vague

threshold towards photo-realism.

Functional realism is about providing the same visual information as found in reality. The main concern of this type

of realism is about transferring information about the real objects such as their shapes, sizes, motions and materials.

As this type of realism aims at providing the necessary information to perform visual tasks, functional realism can

be found in photo-realistic images as well as in simplistic sketches. Measuring whether functional realism is

achieved is therefore a challenging task as it requires an interpretation from human observers. The functional realism

aimed for in this paper is what engineers would call ‘accuracy’.

Although the three types of realism are compared against “reality” and “real environments”, a realistic 3D

environment is not necessarily one found in the real world. A highly populated city could be displayed photo-

realistically although the buildings and layout have been procedurally generated (e.g. Whelan, Kelly, & McCabe

(2008)). In virtual environments for AEC the common practice is to model existing facilities or facilities under

design or construction. When under design or construction, a valid reconstruction of the future facility is required

before it can be displayed. Several methods and techniques are available to perform these reconstructions. The key

aspect in which they differ is the level of automation. Some methods require a completely manual reconstruction.

Other methods however facilitate that data can be gathered automatically thus leaving everything up to algorithms

which can be run on computers.

1.3 Human and computer effort in the reconstruction of virtual environments

Building a 3D virtual environment based on real world data is a tedious task. In Poelman & Fumarola (2009), we

discussed which technologies can be used and which skills are needed. The question remains to what extent

automation is feasible and appropriate in building virtual environments of real world facilities. This question has

come forward in multiple studies resulting in different approaches. These approaches are mainly situated in one of

the following four groups: geometry based, image based, point cloud based and hybrid approaches (Baltsavias 1998;

Debevec et al. 1996; Hung 2007; Tao 2006; Zlatanova 2008).

In the gaming and animation industry, manual 3D modelling is the dominant approach. This approach, known as

geometry based approach, is one of the most labour intensive ones. It therefore scales badly to large or detailed

environments in which each new object has to be modelled separately. Some level of automation is possible in this

approach, mainly making variations on objects and environments. The skills of the designer are essential for the

quality of the result.

Image based modelling approaches aim at retrieving 3D models from a single or multiple 2D images of a physical

object (Debevec et al. 1998; Hartley and Mundy 1993; Remondino and El-Hakim 2006). The typical workflow starts

156


with the retrieval of (calibrated) pictures. Then geometry is extracted from these pictures. Once the geometry has

been extracted, textures based on the initial pictures are applied. This approach has shown promising results but is

still under research (some recent advances are described in Azevedo et al. (2009), Gruen (2008), and Sinha et al.

(2008)).

The point cloud based approach uses capturing devices capable of recording 3D points in real environments.

Capturing devices are known as active sensors (in contrast to passive sensors, e.g. camera, used in image based

modelling) and laser scanners are a well known example. Active sensors are split in two categories: ground laser

scanners and airborne light detection and ranging (LiDAR) sensors (Hu et al. 2003; Wehr and Lohr 1999). Due to

the advanced equipment and setup, this approach is relatively expensive and difficult to use. The result that can be

achieved are, highly detailed 3D models (e.g. Dellepiane et al. (2008)).

Hybrid approaches use a combination of the aforementioned approaches to surpass some of the disadvantages found

in using a single approach. In most cases, these disadvantages are costs and capabilities of a particular approach.

Although image based and point cloud based approaches are near fully automated, they still require a lot of human

effort. In image based approaches, taking images need to adhere to specific requirements, and postproduction is a

necessity. Point cloud based approaches provide huge datasets which cannot be used directly in virtual

environments. The level of automation is case-specific and approaches are tailored towards requirements coming

forward from that case.

2. CASE STUDIES

2.1 Presenting the case studies

We present four case studies that modelled a real world facility in a 3D virtual environment. The mix of manual and

automated approaches varies between all four case studies: we start with a case study where no automatic generation

took place and end with one that was almost generated automatically. Impressions of the case studies are shown in

Figure 1.

For each case study, we start with a general introduction and the requirements set in terms of what type and level of

reality we aimed for. We continue by discussing the trade offs we had to make in terms of effort, automation and use

of existing data. Finally we discuss the choices made and present how the implementation took place and what the

results were.

2.2 Supervisor training

2.2.1 Introduction

The conventional way of training supervisors for the petrochemical industry combines hands-on training in training

facilities and the use of videos, slides and questionnaires. Experiential learning, as brought forward in serious

games, is assumed to improve the knowledge retention over conventional learning methods. A pilot has been

conducted at a large oil company to explore the benefits of 3D virtual training / serious gaming for training

supervisors in the petrochemical industry. The objective is not to replace current training solutions but to position

serious gaming as an additional teaching method. To do so, we developed an immersive 3D serious game that was

based on an actual training location.

2.2.2 Requirements

The 3D environment is based on an existing location in use for real life training. Therefore a key requirement of the

virtual counterpart was achieving a high degree of photorealism with the existing location. Artificial defects were

going to be integrated and had to blend in the environment.

The following requirements were proposed for the 3D environment:

• The 3D environment has to look photo realistic; spatial correctness is less important than visual

appearance.

• Details that are of importance to the scenario have to be homogeneous to the rest of the environment, so

they will not be easily recognizable.

157


• When possible, use the photographic information gathered during acquisition for the texturing process.

• Weather conditions should be changeable.

FIG. 1 Impressions of the case studies: Supervisor in the upper-left corner, the virtual terminal in the upper-right

corner, the chemical installation in the lower-left corner, and the off-shore platform in the lower-right corner.

2.2.3 Trade offs

Due to the high amount of detail to be generated, we choose a site with a compact size. The site was manually

modelled based on photographs without rectification or orientation. Therefore the models were approximations of

the actual objects with regards to dimensions. Nevertheless they appeared realistic because of the textures that were

based on actual photographs.

Most of the detailing was focused on the objects and locations that were of importance to the scenario. Specific

details take a lot of time to create whereas stock content can be used almost instantly. Therefore the environment

outside the compound (drilling site) was neglected and the buildings that were of less importance were modelled

generically.

Manual 3D modelling was used in favour of automatic acquiring techniques because the “pipeline” for creating the

content for this particular industry and client was just getting started. With no awareness of the manual procedures,

it is very hard to correct mistakes in automatically generated content.

158


2.2.4 Implementation

The environment was implemented in Unreal Engine 31 using the development environment that comes with the

engine. The layout of the site was created with the included editor. The details however, which are harder to model,

were created with specialized 3D modelling tools. As the facility consisted mainly of piping, a component based

approach was created which allowed us to reuse existing models to assemble the complete environment.

Due to the large amount of photographic material used in the environment, approaches had to be found to reduce

memory issues. Procedural textures were therefore used to reapply them on multiple models.

2.2.5 Results

The immersive 3D virtual training environment / serious game demonstrated the power of serious gaming for

training supervisors. Although a thorough evaluation has yet to be performed, the first informal evaluations show

some promising results.

Approximately 40 people provided feedback with regards to the environment and were overall positive. People

familiar with the site directly recognised it up to the details. Most people even assumed the environment was

spatially accurate although it was only approximately correct.

Because manual modelling was the selected method a lot of man hours were used to create the environment. It took

two people 3 full months to just model the environment. The amount of time needed to provide this much detail with

manual methods raises the question if there was no speedier and more cost-effective way to achieve these kinds of

results.

2.3 Automated container terminals

2.3.1 Introduction

Novel modes of operations have to be developed to handle the increasing numbers of containers at modern

terminals. Automated handling equipment provides a solution and is becoming the default choice for the design of

new container terminals across the globe. However little experience and knowledge is available on how to design

this new type of container terminals. Virtual environments provide the possibility to look into the future and

visualize container terminals that do not yet exist. This enables designers to achieve insight, communicate their

designs and experiment with different layouts. We designed a support tool for the design process.

2.3.2 Requirements

The container terminals to be visualized in the virtual environment do not yet exist, ruling out the use of acquiring

technology such as laser scanners. The major source of data consists of the CAD drawings made by designers and

photographs of equipment that will be used in the future terminal.

The virtual environment has to be automatically generated from the CAD drawings. As the virtual environment will

be used throughout the design process of the new terminal, the drawings will be changed and updated regularly,

leading to updates in the 3D virtual environment. This prohibits custom modelling work for the 3D environment as it

will be overwritten with each update.

The following requirements were identified:

Automatic generation of the virtual environment from the layout CAD drawings.

Spatial precision in respect to the CAD drawings.

Precise 3D models of the equipment.

Visual appearance was important for possible presentations.

1 Website at http://www.unrealtechnology.com, accessed on July 14th, 2009

159


2.3.3 Trade offs

To automatically generate the virtual environment, a component based approach was used for the translation from

CAD to VR. The downside of this approach was having the same models duplicated multiple times, resulting in a

decrease of realism as real equipment often differs in details like rust and damage.

Customization based on key features of locations (e.g. environment outside of the terminal) is also not considered in

this approach. As the design process of such terminals takes up to a year (or more), the possibility of developing

custom partial models can be considered. This would give the designers an idea of the real world environment in

which they are operating.


The implementation consists of an Autodesk AutoCAD plug-in and a stand alone application developed using

OGRE for the 3D visualization. The AutoCAD plug-in handles the translation from CAD to VR while the stand

alone application visualizes the 3D environment.

Using a predefined convention based on blocks in AutoCAD, a translation to an intermediate XML format has been

made. This approach, based on the library based approach as described in Whyte et al. (2000), was found to be

appropriate as a large amount of objects are reused throughout different terminals. As the CAD drawings use

millimetres as the main unit, the resulting virtual environment could benefit from this precision.

The stand alone application reads the intermediate XML file to construct the virtual environment. The 3D models in

the virtual environment have been modelled based on CAD blueprints of the actual equipment which resulted in

precise models. Photorealism was further aimed for by using photographs of existing equipment to texture the

models. Photographs were also used for the different types of materials found on a virtual terminal: concrete,

tarmac, metal, etc. The environment was further enriched using aerial pictures.

2.3.5 Results

The resulting virtual environment serves as a support tool in the design process of automated container terminals.

The virtual environment can easily be changed by modifying the CAD drawing. It serves as a knowledge sharing

platform for this novel type of terminals.

Although photorealism is less important than functional realism, the level of photorealism present in the virtual

environment ensures everybody has a clear understanding of the new container terminal. Moreover, using the virtual

environment as a presentation tool helps communicating future developments outside of the company (e.g. fairs and

professional congresses).

2.4 Chemical installation

2.4.1 Introduction

Europe is densely industrialized: the most appropriate locations for industry are taken which leaves little choice for

new industrial sites. Therefore brownfield engineering is more common than greenfield. This implies that knowing

the state of installations is of crucial importance when revamping a site or when maintenance has to take place.

A lot of installations have been designed and built decades ago: if engineering data is available, it is in most cases

too old, incomplete or wrong. Nowadays computer programs are being used to handle 3D engineering data to

support the replacement and design processes. For a proper job, good reference data is essential.

We had a strong need for accurate and complete 3D reference data for a large revamping project. Due to the

complexity of the task at hand the model would have to be ready for interaction. A chemical pipeline of 8 inches

needed to be routed from a storage vessel to a production plant for which it had to cross two pipe racks, a road and

another production facility.

2.4.2 Requirements

The following requirements were identified:

A complete 3D representation of the areas the pipeline has to cross with a tolerance of 2 mm for connecting

flanges and 5 mm for the rest of the model.

160


All crossing structural steel, piping and other objects needed to be present for clash checking.

The individual objects needed to be aligned with the engineering grid.

2.4.3 Trade offs

Laser scanning was chosen over traditional surveying because of the accuracy and completeness that laser scanning

can provide. This means there was some level of automation in the 3D acquisition process. Because the engineering

software that was used in the project could not interact with point clouds, the entire point cloud had to be translated

to solid geometry. The modelling process itself could not be automated at that moment in time.

There were problems in previous projects with cloud-fitted models. In reality most things are a little tilted, offset or

damaged which is reflected in the cloud fitted models. However, most engineering packages handle their models as

being perfect. By bringing in slightly imperfect models into the engineering software things go wrong: e.g. incorrect

centrelines, slopes, and elbows. Because of previous experiences, the decision was made to create the models within

the engineering software with the scans as a reference instead of being depended on cloud fitting geometry

algorithms.


The Leica HDS 30002 was used to acquire 79 scans which were geo-referenced for accuracy and control of the

process. Because the scans were geo-referenced, they could be semi-automatically registered. Although it takes

more time on location, it is for many survey companies the preferred way of working.

Furthermore the scans were modelled by loading them in a piping specific engineering application. They were traced

to acquire a solid model; an approach that is somewhat less accurate then cloud fitting, but the engineering

environment was not able to cope with intelligent objects combined with deviating geometric forms.

The engineering of the model was done by experts: the decisions involving the differences between laser scans and

CAD models could therefore be settled on the spot.

2.4.5 Results

The model was used to interactively create the routing of the piping. By using laser scans as a base for the design

process the time for the entire project has been reduced. Comparing the throughput time with the conventional

planning schemes showed time reduction of approximately 25%.

Less automated geometrical data acquisition would have been difficult because of the divergence and complexity

found in the environment for the pipe routing process. The manual modelling process used in this case study,

resulted in a tedious task to create the 3D models.

2.5 Off-shore platform

2.5.1 Introduction

For this case two methods were used to construct the 3D environment of an off-shore platform. The same part of the

environment was modelled twice: once semi-automatically based on laser scanning and once using fully automatic

modelling also based on the same laser scans. This was done to compare the two results in order to be able to pick

the best result. Part of the results of the automatic modelling process are described by Rabbani and van den Heuvel

(2004). Because this example (laser scanning and automatic modelling) provides insight into an almost fully

automated creation process we discuss the results from a different perspective.

For engineering purposes an accurate 3D model of an off-shore platform had to be generated. Some of the piping

needed to be replaced, preferably a one-on-one replacement. Because labour on an off-shore platform can be more

dangerous and expensive than on-shore, prefabrication of parts is desirable. Moreover the 3D model had to be used

to find the best decommissioning and reattachment of new parts as the piping that needed replacement is

interweaved with the rest of the piping.

2 Website at http://hds.leica-geosystems.com/en/5574.htm, last accessed on July 14th, 2009

161


2.5.2 Requirements

The model had to be generated for engineering purposes; therefore accuracy is of major importance. The piping is

one to three inch in diameter, which is this case means approximately five millimetre accuracy. All objects that

could provide a clash with the decommissioning or reattachment needed to be represented in the 3D model.

An additional requirement was set by the tight schedule the project was on. Due to this, development speed was

preferred over costs, which provided an additional challenge.

2.5.3 Trade offs

Because of the high accuracy standards used in engineering and the tight schedule, laser scanning was the preferred

choice of technology. Conventional surveying could however provide the same models although it would have been

too slow.

Although laser scans can be registered automatically, it was decided to do a manual registration process to have

control over the registration process, because of the uncertainties of automatic methods. For the modelling process

the decision was made to semi-automatically create the 3D model. A fully automatic modelling process followed

afterwards to compare the results.


To get comparable results between semi-automatic and fully automatic modelling, a single room of the offshore

facility was modelled. The room’s approximate length was 12 meters wide, 6 meters long and 4 meters high. There

were 32 scans created to get 80-90% coverage of the room whereas all critical components had 100% coverage.

Control of the scanner and the total station required 2 people.

The semi automatic modelling process was done using region growing algorithms (Rabbani and van den Heuvel,

2004) using specialised point cloud reconstruction software. Solid geometry was generated and exported so it could

be used within a engineering CAD viewer. The viewer was used to interact with the 3D environment so decisions

for spool sizes, decommissioning and fitting could be made between the experts.

2.5.5 Results

The time needed to get the scans done was approximately 4 hours using a Leica HDS 3000. The registration process

took approximately double that time. The registration parameters were all below 5 millimetres. The experts on

location estimated that this would be at last a week to do manually.

The semi automatic modelling process took 15 days to create 2602 objects (planes, cylinders), counting the complex

geometry in planes. In comparison, 2338 objects were detected fully automatically although there were quite some

subdivided objects e.g. round columns could be detected multiple times because of missing information in the point

cloud, slicing up the dataset. The points that were part of the segmentation process were approximately 80% of the

total amount of points whereas the points on the generated objects, 53%. This resulted in 946 planar patches and

1392 cylinders (Rabbani and van den Heuvel, 2004). An estimated 40% of the semi-automated modelled objects

were directly usable.

3. REFLECTIONS AND CONCLUSIONS

The case studies presented show an increase of the level of automation possible in the acquiring and modelling

process for the reconstruction of virtual environments. We started with the discussion of Supervisor which was

modelled manually. We continued with the virtual container terminal which consisted of building a component

library and using this library to automatically construct virtual versions of multiple container terminals around the

world. Thirdly, we discussed a case study wherein a chemical installation was reconstructed using laser scanners but

due to scene complexity, manual models were made based on the point clouds gathered from the laser scanners.

Finally, we presented the case of the off shore platform in which a high level of automation was achieved.

The choices for the different approaches were based on the properties of the real environment and on the goals of the

virtual environment. The real environment can vary from being small to large and from being simple to complex.

The goals of virtual environment can result in the need for photorealism, the need for functional realism, or both.

162


Based on these properties and goals, we will generalize approaches from the case studies conducted. A summary of

these approaches with the properties and goals can be found in Table 1.

Whenever a virtual reconstruction of an environment has to be made, size and complexity of the environment play

an important role. Although automatic techniques give the possibility to acquire spatial information rather quickly,

modelling can be troublesome due to the large amount of data gathered. This data needs to be modelled according to

the possibilities of the visualization engine wherein it is going to be used. When using automatic techniques such as

laser scanning, the resulting point clouds tend to be huge, making it impossible to use it in real-time environments.

To reduce the amount of information, specific algorithms can be used. Nevertheless, these algorithms do not give

optimal result for complex scenes as for the chemical installation case study where a huge amount of pipes and small

details were present. In these cases, manual intervention is still required, reducing the level of automation possible in

the modelling phase. In less complex scenes, algorithms for mesh simplification are better suited for the task.

As using methods for automatic acquiring and modelling is still labour intensive and requires specific skills, the

choice for manual methods is still often considered. Automatic acquiring methods result in a high level of accuracy

(up to a millimetre), thus functional realism, which is not always needed. In training environments, such as

Supervisor, photorealism is preferred whereas accuracy is not even required. In these cases, manual modelling gives

the possibility to fully concentrate on details using a team of artists instead of skilled engineers.

CASE SIZE COMPLEXITY FUNCTIONAL

REALISM

PHOTOREALISM ACQUIRING MODELLING

Supervisor

training

Small Low Low High Manual Manual

Automated

container

terminals

Large Low High High Manual Automatic

Chemical

installation

Small High High Low Automatic Manual

Off-shore

platform

Small Low High Low Automatic Automatic

Table 1 Summary of conclusions from the case studies from the perspective of the properties of the real environment

and the goals of the virtual environments.

Choosing between manual and automatic methods for the acquisition and modelling phase for the reconstruction of

a virtual environment, is not straightforward. Trade offs have to be made based on the environment that is being

reconstructed and the requirements set for the virtual environment. Although automatic methods are gaining

popularity thanks to their accuracy and speed, human intervention is still often needed. Complete automatic methods

can only be used in ideal circumstances, which are less common than one would wish for.

4. ACKNOWLEDGEMENTS

The authors would like to thank APM Terminal Management BV for their input and support on the second case

study.

5. REFERENCES

Arayici, Y., and Aouad, G. (2004). "DIVERCITY: distributed virtual workspace for enhancing communication and

collaboration within the construction industry." eWork and eBusiness in Architecture, Engineering and

Construction, A. Dikbas and R. Scherer, eds., Taylor & Francis Group, London.

Azevedo, T. C. S., Tavares, J. M. R. S., and Vaz, M. A. P. (2009). "3D Object Reconstruction from Uncalibrated

Images Using an Off-the-Shelf Camera." Advances in Computational Vision and Medical Image

Processing, J. M. R. S. Tavares and R. M. N. Jorge, eds., Springer Netherlands.

163


Baltsavias, E. (1998). "A comparison between photogrammetry and laser scanning." Journal of Photogrammetry

and Remote Sensing, 54(2-3), 83-94.

Bouchlaghem, D., Shang, H., Whyte, J., and Ganah, A. (2004). "Visualisation in architecture, engineering and

construction (AEC) " Automation in Construction, 14(3), 287-295

Debevec, P., Taylor, C. J., and Malik, J. (1996). "Modeling and rendering architecture from photographs: a hybrid

geometry- and image-based approach." 23rd Annual Conference on Computer graphics and interactive

techniques.

Debevec, P., Taylor, C. J., Malik, J., Levin, G., Borshukov, G., and Yu, Y. (1998). "Image-based modeling and

rendering of architecture with interactivephotogrammetry and view-dependent texture mapping." IEEE

International Symposium on Circuits and Systems, Monterey, CA, USA.

Dellepiane, M., Callieri, M., Ponchio, F., and Scopigno, R. (2008). "Mapping Highly Detailed Colour Information

on Extremely Dense 3D Models: The Case of David's Restoration." Computer Graphics Forum, 27(8),

2178-2187.

Ferwerda, J. (2003). "Three varieties of realism in computer graphics." Human Vision and Electronic Imaging VIII,

T. N. Pappas and B. E. Rogowitz, eds., Santa Clara, California, USA.

Gruen, A. (2008). "Reality-based generation of virtual environments for digital earth." International Journal of

Digital Earth, 1(1), 88-106.

Hartley, R. I., and Mundy, J. L. (1993). "Relationship between photogrammmetry and computer vision." Integrating

photogrammetric techniques with scene analysis and machine vision, Orlando, Fl, USA.

Hu, J., You, S., and Neumann, U. (2003). "Approaches to large-scale urban modeling." IEEE Computer Graphics

and Applications, 23(6), 62-69.

Hung, Y.-P. (2007). "An Image-Based Approach to Interactive 3D Virtual Exhibition." Lecture Notes in Computer

Science, 4872, 1.

Kam, C., and Fischer, M. (2004). "Capitalizing on early project decision-making opportunities to improve facility

design, construction, and life-cycle performance: POP, PM4D, and decision dashboard approaches."

Automation in Construction, 13(1), 53-65.

Poelman, R., and Fumarola, M. (2009). "Technology and skills for 3D virtual reality serious gaming: "Look before

you leap"." 40th International Conference of the International Simulation and Gaming Association,

Singapore.

Rabbani, T., and van den Heuvel, F. (2004). "Methods for Fitting CSG Models to Point Clouds and their

Comparison." Computer Graphics and Imaging.

Remondino, F., and El-Hakim, S. (2006). "Image-based 3D modelling: A review." The Photogrammetric Record,

21(115), 268-291.

Shiratuddin, M. F., and Thabet, W. (2007). "Information Rich Virtual Design Review Environment." 4th W78

Conference, Maribor 2007 & 5th ITCEDU Workshop & 14th EG-ICE Workshop.

Sinha, S. N., Steedly, D., Szeliski, R., Agrawala, M., and Pollefeys, M. (2008). "Interactive 3D architectural

modeling from unordered photo collections." ACM Transactions on Graphics, 27(5), 1-10.

Tao, C. V. (2006). "3D Data Acquisition and Object Reconstruction." Large-scale 3D data integration, S. Zlatanova,

S. Zlatanova, and D. Prosperi, eds., CRC Press.

Wehr, A., and Lohr, U. (1999). "Airborne laser scanning - an introduction and overview " ISPRS Journal of

Photogrammetry and Remote Sensing, 54(2-3), 68-82.

Whelan, G., Kelly, G., and McCabe, H. (2008). "Roll your own city." 3rd international conference on Digital

Interactive Media in Entertainment and Arts, Athens, Greece.

Whyte, J., Bouchlaghem, N., Thorpe, A., and McCaffer, R. (2000). "From CAD to virtual reality: modelling

approaches, data exchange and interactive 3D building design tools " Automation in Construction, 10(1),

43-55.

Woksepp, S., and Olofsson, T. (2006). "Using virtual reality in a large-scale industry project." The Journal of

Information Technology in Construction, 11, 627-640.

Zlatanova, S. (2008). "Data collection and 3D reconstruction." Advances in 3D Geoinformation Systems, P.

Oosterom, S. Zlatanova, F. Penninga, and E. M. Fendel, eds., Springer Berlin Heidelberg.

164


EVALUATION OF 3D CITY MODELS USING AUTOMATIC PLACED URBAN AGENTS

Gideon D. P. A. Aschwanden,

Chair for Information Architecture, Switzerland;

[email protected]

Simon Haegler,

Computer Vision Laboratory, Switzerland;

[email protected]

Jan Halatsch,


[email protected]

Rafaël Jeker,


[email protected]

Gerhard Schmitt, Prof.,


[email protected]

Luc van Gool, Prof.,


[email protected]

ABSTRACT: We present a method for populating procedurally generated 3D city models with crowds of artificial

agents. It is targeted towards the analysis, prediction and visualization of occupant behaviour in urban planning.

We simulate and quantify correlations on the following aspects: functions of buildings, number of people and

fluctuation in density. Potential practical applications are for example a) to determine bottlenecks in public transit,

b) to identify possible problems for evacuation scenarios, c) to evaluate the demand for and the accessibility of

amenities as well as d) the stress of pedestrians to evaluate quality of life indicator for a given urban region . The

occupants’ location data – represented by the agents - and relevant semantic metadata are encoded inside a

grammar-based city modelling system. This information is used for the context-dependent automatic placement of

occupant locators during the procedural generation process of the urban 3D model. Most of the underlying

parameters are interconnected with each other. For example, the number of resulting agents corresponds to the size,

function and location of one specific building. Once a 3D city model has been generated, occupants are represented

by agents using a) a commercial fuzzy logic system and b) pre-animated 3D avatars. The agents find their way

through the city by moving towards points of interest to which they are attracted. Each individual agent draws

specific paths while interacting with the urban environments and other agents. Every path describes a set of

parameters, for example speed, space available and level of exhaustion. The ensuing visual diagrammatic

representation shows the resulting agent paths in correlation with the virtual environment. This offers the

opportunity to investigate parts of a city and optimise corresponding aspects with minimal interventions on the

urban level. We show the application of this method to evaluate planning interventions in the urban fabric and

monitor the correlating effects.

KEYWORDS: Artificial intelligence, agent-based, crowd simulation, Space Syntax, urban planning, design

evaluation, occupant movement.

165


1. INTRODUCTION

The number of planners who come to the conclusions that urban plan should be more aligned to the needs of the

pedestrian occupant is increasing. It seems quite clear that the more an urban design works out as a performing

environment for pedestrians including infrastructures such as public transport or pedestrian miles the less

consumption of energy and emission of CO2 will be performed in private transportation. Therefore we like to

present a new approach to evaluate visually how urban designs perform in a pedestrian context. Our method

combines the robust techniques of commercial crowd simulation applications with the iterative nature of procedural

city modelling techniques in order to have a) fast as well as understandable visualizations and b) a effective design

tool at hand that enables the designer to perform the comparison of design alternatives very quickly. This method

offers as well an added value for the entertainment industry while delivering high quality imagery output through

standard production pipelines and a decreasing workload the procedurally generated urban layouts that are simulated

as a potentially realistic urban environment with associated virtual occupants. Traditionally, costs and time needed

to produce populated digital urban sets are enormous for movie, game and interactive VR projects.

1.1 Related Work In the following we give a short outline of selected works related to the simulations of groups and crowds. For a

more comprehensive description we refer the reader to (Magnenat-Thalmann et al., 2004). Models for group

behaviour have been an active research field since the late 19th century, for example LeBon (1895). Today’s

computer simulation models have a relatively young history. Most relevant approaches have been realized within the

last 20 years and are specialized as a specific solution for different fields.

More specifically is Reynold’s (1987) flocking method, which uses particle systems and represents one of the most

common approaches for simulating group movement within the entertainment industries. Brooks (1991) provides a

comprehensive foundation on which many of the recent agent models and theories are based. He describes many

failing artificial intelligence approaches to set-up intelligent agents. Musse and Thalmann (2001) introduced a more

flexible model with hierarchical behavior. Physics and body effects had been described by Helbing et al. (2000) to

simulate escape panics effectively. In other fields like robotics (Molnar and Starke 2001), safety science (Still 2000)

and sociology (Jager et al. 2001) similar approaches have been created simulations involving groups of individual

intelligent units. Despite the actual size of the crowd simulation research basis interdisciplinary exchange between

groups of researchers is relatively limited. Hillier and Hanson (1984) introduced the idea that a city and spaces in

general can be divided into components to analyze them as a network of choices and be represented as maps and

graphs. Penn and Turner (2001) then described their method to use urban agents within their space syntax system.

Parallel systems evolved in computer graphics to populate urban environments in real-time (Penn und Turner, 2001;

Tecchia and Chrysanthou, 2000). In Aschwanden et al. (2008) we introduce our occupant simulation method, which

is now extended by the present work. The presented procedural modelling technique has been initially presented by

Parish and Mueller (2001) for the modelling of cities, by Mueller et al. (2006) specifically for the modelling of

buildings and has been practically applied to the context of urban planning by Ulmer et al. (2007) and Halatsch et al.

(2008a and 2008b).

2. URBAN MODEL

The authors interpret the term ‘the city’ as a complex, distributed, interconnected and rapidly changing system that

can be currently understood as a fabric of space depending on a) People, b) Function, c) Space, and d) Physical

Environment

Each aspect listed above is typically under the influence of a number of variables and forces. As urban planners, our

aim is to try to control both the layout and the functioning of the urban system, while regarding the most influential

variables of a given urban situation through our empiric planning knowledge. In the case of a complex city space,

the occupants may either accept the designers’ work or not, and their subsequent use and acceptance of the designed

urban space defines a successful urban planning. In urban planning there exist many important correlations. For

example, on one side economic demand asks for more high-density buildings in city centers, which coexists with the

commuter’s demand to have access to green open spaces. These contradicting desires call for an equilibrium, which

must be recalculated constantly. Our goal is to satisfy both; the economic and personal needs of people. It requires

us a) to understand the given situation, b) to indentify the necessary semantic metadata, c) incorporate insights into

166


the procedural 3D model and the agents’ model, and d) to run the simulation. On a very abstract level, the output

presents vital clues about possible human acceptance of the given configuration of a city. The quantifiable judgment

of the agents, while rather simple and predefined, seems to be very accurate. We are not implying that one agent

alone can provide us with all of the information we need, but by combining hundreds or thousands of agents within

the one setting we create a kind of group intelligence.

3. URBAN DENSITY

3.1 What Kind of Density?

Urban densities are under constant discussion - the factors by which to classify density are not clear. In addition, the

output value does not have a clear unit. Some urban planners find that the number of occupants per square kilometer

is a useful figure. Another useful figure may be the number of cars recorded between urban centers, but these two

figures have already very little correlation. A major achievement is to have a correlation between the built

environment, number of people and traffic. Our contribution is the enhanced level of detail, in contrast to the

aforementioned examples of density; we use the existing plots in combination, with those being developed, and take

note of buildings already erected on these plots to predict the number of people likely to be occupying that area.

Other calculation methods, like Arnott and Small (1998), use distant functions from a virtual centre or a grid overlay

on the city. This roughness allows arbitrariness in both of these methods.

Our approach (see figure 1) is pragmatic, because we want to know how many people are coming out of the building

and are walking the streets. There are several aspects involved; three of them have a major influence: a) the floor

size of the building, b) use function and c) the location of the building. These factors enable us to predict, how many

people are coming out of a building at a certain moment in time. Verified statistical data in a reasonable level of

detail is key to comprehensive results. Until now we only have input data from Switzerland what is the limitation to

our approach.

FIG. 1: Human Locator (Input-Output based calculation method for the prediction of the number of people coming

out of a building).

3.2 Calculation4

3.2.1 Source of data

We used the Swiss census data by the Swiss government to define the floor space per person in a building living

area as well as the space requirements for offices and workspaces. The overall number of occupants is not decisive

enough, only in combination with the fluctuation over time were able calculate the people moving in and out of the

building. These factors are very close related to the characteristics of a place. We automated the calculation and

specified it according to every commune in Switzerland (Diener, 2006). This was necessary due to the high disparity

of the input data.

3.2.2 Example of a residential house

For residential areas we used the floor space of the apartments and divided it by number of residents to get the

average space available to every resident. With normal fluctuations arousing from people moving, the housing

167


market is not operating at full capacity. Therefore an economical approach (Wüst und Partner,

http://www.wuestundpartner.com/) is also incorporated. The combination of net floor space, space available,

location and the fluctuation over time allowed us to predict the traffic produced by every building. The limits of

such automated calculation is reached in cases where the traffic generated is not a stable function, such as a stadium

emptying itself with 30 min 2 times a week or when the number of buildings is to small to balance out the statistical

imprecision.

3.3 Preliminary Findings

The available average floor space for residents is varies in Switzerland from 35.5m2 (Zürich) to 55.7m2 (Maisprach).

Both numbers are higher then the space provided for office workstations from a minimum 4.46m2 for secretarial

workstations to 27.89m2 for a vice-president office. This shows that the traffic generated by office buildings is much

higher then the residential buildings in general and is also more volatile over time. The 3rd places, such as

restaurants, cinemas as well as hospitals, produce an even higher number within their peak time but are difficult to

predefine.

4. AGENTS

The agents that we used consist of perceptual, behavioral and cognitive components in combination with a physical

body to test the practicality of our 3D city model. Every agent represents a measuring device by walking in the city

using its function channels to learn about the environment and the artificial brain to process it and give us

information about the path it took. The agents are able to see, hear its environment and read information that has

been ‘written’ to the ground of the 3D environment. These abilities are processed inside the artificial intelligence

logic by each agent’s instance that is present in the 3D environment for evaluation. This enables the agents to

interact with their surrounding, their built environment as well as with other agents. A unique set of personality

characteristics and abilities for each agent allows this collection of individual bodies to be a realistic representative

of a standard population. The agents’ entity of body and brain is not only a shape; it merely is a semantic organism

enabling the agent to interact with distinct features and abilities.

4.1 Sensory Channels for Perception

We are using multiple sensors as receptors for the agent to understand its environment. Signals from a single object

are affiliated by a sensory system. Like humans the agents have a limited set of senses through which to experience

the environment. The following steps of filtering, selection, rating and simplification have been incorporated into the

brain. Each of these artificial senses allows the agents to operate within and interact with their surroundings:

Read the ground: Because the agents are able to understand where they walk, this affects the way they interact with

their environment. We trained them to avoid streets, prefer to walk on the sidewalk but not to neglect streets for

crossing. In addition, we were even able to train the agents to have the ability to understand a whole set of colors.

This gives us the freedom to underlay our 3d model with a master plan readable to the agents.

Vision: As simplified replicas of humans, the agents are strongly controlled by their visual input. We use vision to

detect collisions, which is an overruling factor (Gibson, 1979). A change in color of an agent is used to

communicate its state to other agents. For example we express human emotions as a color code, allowing typical

emotions to be conveyed in situations such as when a collision occurs. Red, representing anger, or blue, representing

indifference, are visible in the agents.

Hear: As in reality, when we can detect the distant chiming of church bells, the ability to ‘hear’ allows the agents

to detect stimuli, which are out of sight. Agents us the frequency and the amplitude of the noise to guide them

towards distant locations or points of interest which are out of sight.

4.2 Set of Goals

Several analytical tools have been presented which examine the correlation between how people move within, and

use, the urban environment (Penn and Turner, 2001). In our approach every agent has its own set of preferences in

relation to the specific places he wants to visit. For example the desire to stop for coffee in the morning on the way

to work may be common, but certainly is not representative for all commuters. In dynamic environments, such as

168


busy city streets at rush hour, the agent is able to adjust its path and goal accordingly. For example, the agent can

change its mind in real time. Each possible destination has a maximum occupancy which, when reached, reduces its

attraction for agents. Upon arrival, an agent can differentiate between ‘buying a cup of coffee’ and ‘having a whole

meal’, and therefore spends time accordingly. The goals are set accordingly to the master plan, which consists of

function layers, such as living, working, industry etc., and creates an additional set of locations for special activities.

This is where the agents would typically spend time between work and home. By setting the master plan, every

urban planer has an idea where special activities are likely to take place. This would then either promote further

activity, or calm the surroundings. By compiling such information we reduce buildings to symbols of their function.

This allows us to focus on the elements that are frozen in the master plan, such as the width of the street and

sidewalk, the orientation of the doors and their size, and most importantly the use factor.

4.3 Output

During the simulation, every agent draws an individual color-coded line on the ground. The color change, triggered

by the brain, is representative of both his emotions and his analytical experience of the environment. Every agent is

entirely unique. The artificial intelligence conveys information about the effort it took the agent to reach the

designated points of interests, if his journey was too time consuming, and how much personal space was available to

him during this time. If a significant number of agents appear to be exhausted at the very same spot, we know that

we have to adjust something. Depending on the location a rest area with benches may be needed. Also, if the net

travel time appears to be too long we assume that the agents would rather take public or private transport then walk.

With this information we can adjust the stops of the public transportation system to increase the time efficiency for

the majority of occupants.

5. URBAN ENVIRONMENT

The workflow to create a virtual urban model for crowd simulation is similar to the workflow used by architects in

urban planning. Based on a number of maps (color-coded images and other rasterized data), we create street-

networks and rough volumetric models of the buildings. These volumes are used to distribute urban functions and

building density and guide the agent navigation as described in the previous section. Based on these rough building

volumes, we create the final detailed geometry needed for visualization and the low-polygon geometry used in

simulation. The original aspect of our workflow is the use of procedural modeling to automate the creation of street

networks and building geometries. A big advantage of this is the adaptability - the urban environment is able to

change for each simulation without redrawing from scratch. We automate the export of control data for the

simulation as well as the visualization, which differ radically. They both are stored inside the same procedural scene

description and will therefore always be in sync. The following sections describe our workflow in more detail.

5.1 Creating the Urban Layout

FIG. 2: Color-coded attribute maps are used to control the generation of street-networks and building geometries.

From left to right: topology, obstacles, height map and building heights (skyline).

We use a futuristic scenario in the area of Zurich, Switzerland, to describe our workflow. In this example, we use

several maps (see figure 2) to encode aspects of the urban layout (e.g. topography, elevation, obstacles, skyline, land

usage). To create the street network, we have two options: (1) generate a generic, procedural, rule-based network

which follows the attribute maps (e.g. along the color gradients) or (2) import a network from a vector data source

(e.g. from openstreetmap.org). For simplicity, we show a generic street network (see figure 2, left/middle). The

algorithm is an extension of the L-system based street network generator described in Parish and Mueller (2001).

169


The building plots can be created in a similar way: (1) by a generic subdivision algorithm or (2) by importing a

vector data source. Again, we show generic parcels (figure 3, right), also created with the algorithm described by

Parish and Mueller (2001). Once the building parcels have been generated, we are ready to apply the procedural

rules to create the building and street geometry.

5.2 Procedural Modeling using Shape Grammars

Most recently, research in architecture (and subsequently computer graphics) has produced a number of production

systems for architectural models, such as Semi-Thue processes, Chomsky grammars, graph grammars, shape

grammars, attributed grammars, L-systems or set grammars (Vanegas et al., 2009). All of these methods expose

different application possibilities and levels of efficiency to the user. The shape grammar concept has recently been

made more applicable to computer graphics and daily usage (Mueller et al., 2006) and is now commercially

available in the package “CityEngine”, which is used in this paper (Procedural Inc., http://www.procedural.com).

The key tools of the context-sensitive shape grammar as implemented in the CityEngine consist of: shape operations

for mass modeling, component splits for transition between mass and facade, split operations for building facades,

spatial query operations and more.

In this section, we show how the shape grammar, as implemented in the CityEngine, is used to model some simple

building volumes for the Zurich scene. Each building lot (parcel) is assigned a shape grammar rule set. A rule set

consist of production statements in the form:

Predecessor → [case Condition1:] Successors 1 [case Condition2:] Successors 2 [else:] Successor N

With this the successors can be composed of several shape operations and query statements. Listing 1 contains a

small example rule set and figure 3 shows a possible result. Figure 4 shows the result of the application of the

example rule set to the complete scene.

// get building height from the “skyline” map

// intensity values of the red-channel are linearly mapped from

0..1 → 10..200

attr height = map_01(red, 10.0, 200.0)

// the first rule checks for rectangularity

Lot --> case geometry.isRectangular(10) : case scope.sx-10 < scope.sz && scope.sx+10 > scope.sz:

RectQuad(height)

else: … other rules … else: … other rules …

// scale the building and center it on the lot

RectQuad(h) --> s('.8,'1,'.8) center(xz) extrude("y", h) UShaped

// divide along the x axis into two wings

UShaped --> split(x){'rand(.3,.5): Facades | ~1: SideWings}

// subdivide again along the other horizontal axis

SideWings --> split(z){'rand(.3,.45): SideWing | ~1: NIL | 'rand(.3,.45): SideWing}

SideWing --> 30% : Facades

30% : split(x){ 'rand(0.2,0.8) : Facades}

else : s('1,'rand(0.2,0.9),'1) Facades

// the generation stops with the 'Facades' shape

Left: FIG. 3. A simple U-shaped mass model consisting of three volumes (shapes).

Right: Listing 1. This "CGA shape" source code produces simple U-shaped mass models consisting of three shapes.

Some parts have been omitted for simplicity.

170


FIG. 4: On the left, the resulting scene is shown with the same simple set of shape grammar rules applied to all

building lots. The figure on the right shows a close-up of the high-rise area in the center.

5.3 Generation of the Control Data for Crowd Simulation

In Section 4.1 discussed the input channels (e.g. ground colour, vision, sound) used by the agents in the crowd

simulation. In the current section, we describe how these channels are derived from the procedural urban model. In

other words: we need to extend our grammar rule set with an optional set of rules which trigger the generation of

input data for the simulation. Our crowd simulation setup needs three types of input:

- Simplified building and street geometry to visually guide the agents in order to prevent collisions.

- Locators for initial agent placement and points of interest to implement building functions.

- Terrain geometry with a texture map where additional color-coded control data is stored. For example, we are

using color intensity in the door areas to encode the number of people that are entering or leaving a certain

building.

FIG. 5: Illustration of the data-flow between the CityEngine (left) and the crowd simulation tool (right). Note that

all the necessary input data to the simulation has been generated automatically from a single grammar rule set.

The CityEngine contains an export algorithm which scans the names of the terminal shapes of the grammar

generation process (in the U-shaped building example above these were called 'Facades') and triggers the creation of

one of the input data types for the crowd simulation based on a set of name patterns. For a typical agent simulation

scenario the relevant terminal shapes are usually called 'door', 'window', 'sidewalk' etc. Figure 5 depicts the data-

flow between the CityEngine and the crowd simulation tool. The necessary modifications to an existing grammar

rule set look as follows:

1. Based on the simulation scenario and the tool (in our case “Massive”), choose the appropriate control

features (vision, sound, colours, vector fields, etc.) to trigger certain behaviours of the agent (walk, stop,

attract, etc.).

2. Identify the parts of the urban environment with which the crowd agents interact (e.g. doors, windows,

sidewalks etc)

171


3. Introduce rules, which trigger certain simulation features not necessarily visible when rendering the scene

(e.g. a locator on the bottom face of a building entrance). The main idea is to make these rules conditional

on a rule attribute to be able to comfortably switch between model generation for visualisation and model

generation for simulation.

4. Also introduce conditional rules to deactivate all geometry not needed in the crowd simulation.

For our final results, the previously mentioned modifications resulted in the following encodings:

• Locators for agent sources: all terminal shapes called 'door' trigger the generation of a locator and a color

on the bottom-face of the doors. The color is used to encode the amount of agents emitted from this door.

• Locators for points-of-interest: the doors also trigger a sound source, which is connected to it to attract

agents. We distributed 10 different frequencies for the ten different POIs (POI0,…,POI9) according to the

master plan. If necessary, the distribution of the POIs can also be controlled by grammar rules.

• Locators for “background” agent sources: the shape grammar splits the sidewalks into small stripes and

assigns agent sources to them according to a certain probability (see Figure 6).

FIG. 6: From left to right: (1) detailed geometry used for visualization. (2) The rough volumes used for agent vision

in the simulation. (3) The blue/red marked areas show the 'door' terminal shapes used to generate agent source

locators. (4) Volumes (gray), color-code (red) and locators (yellow) imported into the simulation tool ("Massive").

6. EXAMPLES

To prove our concept we are presenting two case studies. The first aims to push the ability of our method to the limit

by using a large number of buildings and many agents. With the second case study we use a higher level of detail.

6.1 Dübendorf

Dübendorf, a suburban centre of Zürich, currently houses a military airport, which will be made redundant. The area

in question is the biggest to be developed in Switzerland in the near future. With an increasing trend of living in

suburban areas there is increased pressure to develop the area into a compact new centre for Dübendorf and

surrounding areas. In this case we can show the density differences between the area of high-rise buildings in the

centre and the low-density living quarters on the outskirts. There is no absolute rule about density or traffic - for

some attractions an increased flow of occupants is favored, while for others this is not the case. We show in figure 7

the layer of personal space. The problem we identified here was that the entrance / exits of the buildings were

opposite to each other. They would be better placed in an asymmetric way to avoid collisions.

172


FIG. 7: Left: Quantified visualization of 6800 agents inside the given area. Middle and right: Occupant stress

analysis; a very intense magenta represents not enough personal space.

To handle the amount of agents in this case, we group agents into three major groups (Jager et al., 2001). The group

who are coming from home (GotoWork), the group who come from work (GotoHome) and the group who have no

specific goal to reach and occupy the streets as a present background noise.

6.2 Second Case

The setup consists of a pedestrian crossing with different typical activities, and a street, which represents an obstacle

to the agents. This time the agents find their way from their source to their final destination by making a selection

from a list of possible secondary goals and marking their way within their individual path. As a result we can see

increased traffic around the high-rise buildings as well as the crossing of the streets in the north (see figure 8). As

seen in the collision map, there is no stress problem at street crossing. However, even with the same amount of

pedestrians, there is a problem in the middle of the pedestrian walk way. This is due to several reasons. One reason

is the agents flocking behavior, where agents align themselves after increasing amount of iterations. Additionally, if

an agent is making decisions in that particular area it is likely that he will change his goals. Interestingly, the agents

do not favor the two asymmetric pedestrian crossings. In order to find the next crossing they can easily block each

other’s path. The total-time-spend map reveals the need for another public traffic stop at the south end to even out

the time needed to reach all goals.

FIG. 8: High-density crossing; from left to right: Density, collision map, total time spend to complete the list.

7. RESULTS

Our simulation method enables the urban planner to adjust the design from small scale to large scale. The high-

resolution pictures (figure 9) are also a method of communication to all stakeholders used to discuss ideas and

address problems. With an increased number of decisions made unilaterally, decision making time and meeting

requirements are reduced.

FIG. 9: High-density crossing; from left to right: Density, collision map, total time spend to complete the list.

173


With a limited amount of resources, this allows the urban planer to choose the smallest possible intervention to gain

the largest output. Small obvious changes are more likely to be accepted by the occupants and therefore have a

higher chance of creating a permanent effect on the city. Several owners adjust their house entrances after some time

to optimize their property, or alternatively a change in ownership occurs triggered by an increase in value and a need

for more area. In the design stage, these changes can be simulated and taken into account.

8. DISCUSSION

Even with useful results that are relatively easy to understand, we have to research possible deviations between our

prediction and reality. Despite reliable statistics, it is virtually impossible to reconstruct the decision process of

human beings. For this reason we concentrate on the path finding abilities of the agents and rely on predefined goals.

Another limitation of our method is the calculation time. The enhanced complexity of the urban layout increases the

calculation time exponentially.

9. FUTUREWORK

We have not yet been able to automate the input from the agents into our generic city model. It is possible to

totally automate this evaluation method with a combination of python scripting and visualization toolkits like

RenderMan from inside the CityEngine as well as from inside MassivePrime. Another step would be to adjust the

buildings and public transport networks to be autonomous and to be outfit with intelligence as well. The adapting

environment within our simulation breaches the current gap that exists because of the differences in time scale and

explains how sudden events form the city.

10. AKNOWLEDGMENT

We would like to thank the participants of the elective course ‘Collaborative City Planning’ at the Chair for

Information Architecture, spring semester 2009, who designed the model of the case studies as well as Martina

Jacobi who did the preprocessing work for the simulation. Especially, I would like to thank Alice Vincent for her

tremendous involvement in the writing process.

11. REFERENCES Arnott R. und Small K.A.: 1998, Urban Spatial Structure, journal of Economics Literature, Vol. XXXVI, Pages:

1426 - 1464

Gibson J J, 1979 The Ecological Approach to Visual Perception (Houghton Mifflin, Boston, MA)

Diener, R., Herzog, J., Meili, M.,de Meuron, P., Schmid, Ch, (2006). "Switzerland - An urban Portrait." ETH Studio

Basel: Conteporary City Institute, Pages: 64-109.

Jager, W., Popping, R. and van de Sande, H.: 2001, Clustering and Fighting in Two-party Crowds: Simulating the

Approach-avoidance Conflict, Journal of Artificial Societies and Social Simulation, 4(3).

LeBon, G.: 1895, Psychologie des Foules, Alcan, Paris.

Halatsch, J., Kunze, A., Burkhard, R., and Schmitt, G. (2008a). ETH Value Lab - A Framework For Managing

Large-Scale Urban Projects, 7th China Urban Housing Conference, Faculty of Architecture and Urban

Planning, Chongqing University, Chongqing.

Halatsch, J., Kunze, A., and Schmitt, G. (2008b). Using Shape Grammars for Master Planning, Third conference on

design computing and cognition (DCC08), Atlanta.

Magnenat-Thalmann, N. and Thalmann, D. (eds.): 2004, Handbook of virtual humans, Wiley, Chichester.

Müller P., Wonka P., Haegler S., Ulmer A. and Van Gool L. 2006. Procedural Modeling of Buildings. In

Proceedings of ACM SIGGRAPH 2006 / ACM Transactions on Graphics (TOG), ACM Press, Vol. 25, No.

3, pages 614-623.

Musse, S. R., Thalmann, D.: 2001, Hierarchical Model for Real Time Simulation of Virtual Human Crowds, IEEE

Trans. on Visualization & Computer Graphics, 7(2), pp. 152-164.

Parish Y. I. H. and Müller P. 2001. Procedural Modeling of Cities. In Proceedings of ACM SIGGRAPH 2001, ACM

Press / ACM SIGGRAPH, New York. E. Fiume (ed), COMPUTER GRAPHICS, Annual Conference Series,

ACM, pages 301-308.

Penn, A. and Turner, A.: 2001, Space Syntax Based Agent Simulation, in M. Schreckenberg, S. D. Sharma (eds.),

Pedestrian and Evacuation Dynamics, Springer-Verlag, Berlin.

174


Procedural Inc., Zurich, Switzerland; http://www.procedural.com

Still, G. K.: 2000, Crowd Dynamics, PhD thesis, Warwick University.

Aschwanden, G: 2008, Agent-Based Crowd Simulation for Urban Planning, In Proceedings of eCaaDe 2008.

Tecchia, F. and Chrysanthou, Y.: 2000, Real-Time Rendering of Densely Populated Urban Environments, in Proc.

Eurographics Rendering Workshop.

Vanegas, D. Aliaga, P. Müller, P. Waddell, B. Watson and P. Wonka. 2009. Modeling the Appearance and Behavior

of Urban Spaces. State of The Art Reports Eurographics 2009.

175


INTEGRATION OF AS-BUILT AND AS-DESIGNED MODELS FOR 3D POSITIONING CONTROL AND 4D VISUALIZATION DURING CONSTRUCTION

Xiong Liang, PhD Candidate

Department of Civil and Structural Engineering, Hong Kong Polytechnic University, Hong Kong, China;

[email protected]

Ming Lu, Associate Professor,

Department of Civil and Structural Engineering, Hong Kong Polytechnic University, Hong Kong, China;

[email protected]

Jian-Ping Zhang, Professor,

Department of Civil Engineering, Tsinghua University, Beijing, China;

[email protected]

ABSTRACT: This paper describes an on-going research on as-built and as-designed model integration and

visualization to support critical physical tasks during construction such as positioning components during

installation and inspection of the geometric dimensions, the layout and the orientation on as-built components. The

objective of this research is to develop a generic, extensible framework to support the above integration and

visualization processes. Critical issues, principles and challenges arising from the development of the framework are

discussed. Some preliminary results are presented in a case study to demonstrate the usefulness of the framework in

terms of leveraging data collection technologies and visualization technologies to improve the current construction

practices. The proposed framework is supposed to provide a cornerstone for developing the next generation 4D

system, which will be used not only for project management tasks but also for physical tasks during construction

engineering operations.

KEYWORDS: Visualization, As-built and As-designed Integration, 4D CAD, Construction tasks.

1. INTRODUCTION

The 4D technology and its application in architecture, engineering, and construction (AEC) have been intensively

studied in the areas of construction planning (Liston et al. 1998), constructability review (Hartmann and Fischer

2007) and site layout planning (Ma el. 2005) etc. 4D technology has improved many practices in the construction

industry. For example the construction planning is improved through visualizing the animated 3D construction

sequence which leads to better communication among stakeholders involved, enhanced visibility and clarity of

project or process plans, more dynamic and integrated review of different plan options.

However, to date, the 4D technology has not yet been accepted on a large scale in construction practices. There are

many reasons to account for such hesitation: The complexity of 4D modeling and the lacks of integrated tools require

talents with special skills (Haque and Mishra 2007); Barrett (2000) attributed it to the dynamic, fragmented and

combative nature of the construction industry. In our past experiences on 4D application in construction projects in

China, the lack of tangible support to contractor’s critical tasks is identified as an important factor that has hindered

practitioners to adopt the 4D technology during construction execution. We enlisted many project managers’ views

on 4D technology, one common opinion is that the cost of 4D modeling is very high, however, it is only used on one

or two occasions in the early stages, e.g. to facilitate the communication of contractors’ bid proposals with owners.

Then the 4D models are shelved in the cabinet without follow up applications. This observation is also echoed by

Hartman et al. (2008) who analyzed twenty-six case studies of 3D/4D model applications on construction projects.

They discovered that practitioners mostly used the models in one application area only. They suggested that further

177


research on the integration of 3D/4D model technologies into work and business processes of project teams should

address the more widespread use of 3D/4D models throughout the lifecycle of a project.

At present, research of 4D technology and its application largely focus on management tasks such as presentation

and communication of project plans and construction methods. There is little research reporting the use of 4D models

during the construction execution phase to support physical tasks like positioning components during installation and

inspection of the geometric dimensions, the layout and the orientation on as-built components. Bernold (2002) also

stated that modern design software is able to digitally model building elements in their spatially correct

configurations, and without interference. However, when it comes to the setting of actual construction in the field,

those spatial models have not yet found much value-added applications.

On the other hand, reality capture technologies have matured sufficiently to be utilized for as-built data collection on

construction sites. Many advanced positioning and data collection technologies (GPS, total station, laser scanner) are

being gradually adopted by construction practitioners. The feasibility and suitability of combining these advanced

technologies with computer-based visualization technology to improve current practices in the construction industry

have been studied by many researchers. Bernold (2002) described their work on digitally merging spatial design data

with the digital model of equipment working on implementing the design in support of site operation. Akinci et al.

(2006) combined automated data capture technologies (laser scanner, temperature sensor) with designed project

models for quality control on construction sites. In recent years, augment reality (AR) which superimposes computer

generated images over a human’s view of the real world has been studied to demonstrate its feasibility and to

validate its suitability in the construction industry (Shin and Dunston 2008).

Despite the fact that there are many benefits to be materialized from the integration of as-built and as-designed

models in construction, the lack of easy-to-use and integrated tools has hindered the application of the above

mentioned technologies in the construction industry. In this paper we provide a framework for developing the next

generation 4D system that facilitates automated as-built and as-designed model integration and visualization, such

that: (1) the efforts on 4D modelling can be fully utilized throughout the whole life cycle of construction project; in

particular, the physical tasks in a construction project can be improved in terms of quality and efficiency; (2) The

management tasks can be directly linked with the physical tasks so as to enhance project management and control.

We also developed a prototype system to demonstrate the application of the framework to practical construction

scenarios.

2. DESCRIPTION OF THE FRAMEWORK

Shin and Dunston (2008) claimed that a construction process involves various tasks that deal with the creation of

physical structures and elements and most often require visual information to understand and communicate their

complexity and relationship to existing structures or elements. They indentified eight work tasks (layout, excavation,

positioning, inspection, coordination, supervision commenting, and strategizing) may potentially benefit from AR

support. Meanwhile 4D CAD decomposes 3D product models into building components which are linked with

construction activities, some of which can be mapped to the above work tasks. This provides an opportunity to use

the 4D technology to support some daily physical tasks on construction projects if the 4D model is integrated with

real-time site data. This extension to current 4D CAD systems will in turn drive the construction industry to adopt

the 4D technology. However, the modeling and application procedures should be simplified such that construction

engineers would fully leverage the extended 4D tool to cope with construction site tasks.

Fig. 1 illustrates the concept of as-built and as-designed integration and visualization based on the proposed next-

generation 4D technology. In this framework, some critical issues need to be addressed: Firstly, the available as-built

data that may be useful for supporting construction tasks should be indentified and assessed. Secondly, the

decomposition and organization of 3D product models should be considered during 4D modeling processes at

different levels of detail. The next generation 4D system is supposed to be used in both management tasks and

physical tasks throughout the lifecycle of a construction project, so different requirements of the two types of tasks

need to be addressed carefully. Thirdly, the as-built data and the as-designed models need to be correctly registered

in the 3D graphical engine: for example, the two local coordinate systems need to be aligned; The as-built data need

178


to be transformed to meaningful representation in 3D in order to support physical tasks. Lastly, different data

collection technologies have different data structures and different data exchange protocols, which should be

synthesized in an agile way to make this framework extensible and scalable. In the following sections, the above

critical issues are described in detail, and a case study of implementing the framework to support physical tasks

based on a prototype system are presented.

FIG. 1: Overview of the as-built and as-designed integration and visualization framework

3. TAXONOMY OF AS-BUILT DATA

Researchers have studied many types of as-built data for a wide range of application areas in construction, e.g., RFID

and GPS positioning data for resource and material tracking (Ergen et al. 2007; Song et al. 2006); Laser scanner and

temperature sensor data for construction quality control (Akinci et al. 2006); Camera data for enhancing situation

awareness (Katz et al. 2008). It is important to understand what kind of data is suitable for supporting a particular

physical task through visualization. Kiziltas et al. (2008) assessed field data capture technologies for construction

and facility/infrastructure management. Shin and Dunston (2008) identified eight suitable application areas of

visualization technologies in industrial construction. Based on their work, three types of as-built data are selected

that hold promises for supporting construction physical tasks through as-designed and as-built integration and

visualization. The three types of as-built data, their related data collection methods and supporting physical tasks are

listed in table 1.

FIG. 2: Current positioning practices during the installation of steel column

179


TABLE 1: Suitable as-built data types in terms of visualization

As-built Data Data collection method Supporting physical tasks

Point GPS, Total station Positioning, Marking

Point clouds Laser scanners Inspection

Image Camera, Video camera Positioning, Inspection

A point is represented by three Cartesian coordinates X,Y and Z, which is the most important data collected from

today’s surveying practices in a construction project. It is widely used in positioning a target location on a

construction site. Shin and Dunston (2008) indentified that when a positioned element does not require a highly

accurate location, the conventional visual information (marked points or lines) is clear and adequate. However, when

an element needs to be positioned with high accuracy, a marked target location may not be enough. For example,

some elements like steel columns require orientation control during positioning, but it is sometimes hard to indicate

an accurate 3D orientation for a large element in space only with limited target points. In this case, the element being

installed is positioned based on checking a number of reference points or lines, and adjustments to errors on the

position and orientation of the element are made. Workers continuously check and adjust the position and orientation

until the element is placed as per designed. It is a time-consuming loop procedure: fix the element temporarily, check

its location with a measuring device, adjust it, and then check it again. Fig. 2 illustrates the above critical process for

installation of a steel column on a high-rise building. It is obvious that visualizing the points data measured in

relation to the as-designed 3D model will improve the installation process in terms of quality and efficiency.

Point cloud is a set of vertices in a three-dimensional coordinate system, which are generally created by 3D laser

scanners. The point clouds are converted to triangle mesh models, non-uniform rational basis spline (NURBS)

surface models, or CAD models through a process commonly referred to as reverse engineering. Akinci et al. (2006)

studied the feasibility of integrating the point clouds data with a project model for quality control. Raw laser scanned

data contain various types of noises resulting from site constraints. In order to reconstruct the as-built model, the

noises need to be filtered. Furthermore, the point clouds from each scan are represented in the local coordinate

system of the scanner, which makes it impossible to perform direct comparisons with the as-designed model for

defect detection and analysis. Therefore, it is necessary to align all of the scans in a common coordinate system -a

process known as registration. Enabling automatic registration is one challenge that will be addressed in the proposed

4D framework.

An image represents a snapshot of the real world view human sees through a camera, a video camera or a head

mounted display (HMD) device etc. The superimposition of a designed 3D model on the image generates a new

scene that human can’t see directly in the real world, for example, user could see the column that has not yet been

installed as if it were already installed. This computer graphic technology is named as augmented reality(AR). The

AR technology is useful for supporting construction physical tasks. However, the accurate tracking of user’s

viewpoint and orientation is a bottleneck to AR applications due to the lack of tracking technologies that are

functional at the construction site. Behzadan and Kamat (2007) investigated the application of the global positioning

system and 3 degree-of-freedom angular tracking gyros to get the viewpoint and orientation information needed for

accurate registration. Although their method had some limitations such as the accumulation of drift error and errors

inherent in GPS positioning, their work proved the feasibility of using AR technology for supporting the

visualization of construction simulation, and making the simulation more realistic.

4. DECOMPOSITION AND AGGREGATION

4D technology decomposes the product 3D model into components which are linked with construction activities in

order to produce construction process 3D animation. The decomposition mechanism is closely related to the

construction planning, however, the level of detail depends on the planning purpose. When a project manager uses

the 4D tools for project planning, they may not create a detailed schedule for handling each production component.

However, in order to extend the use of a 4D model into production tasks during construction, the decomposition of a

180


product model into the proper production level of detail is very important. For example, in Fig. 3 the 3D model on

the left hand side is a steel component of a high-rise building. It depicts the decomposition of the design-centric

product model prepared by the designer. While the three sub-components on the right hand side represent the

decomposition of the production-centric product model prepared by the contractor.

Different from previous applications of 4D technology in construction planning, this research entails more detailed

decomposition of 3D product models for handling construction physical tasks. As shown in the left part of Fig. 3 a

steel column segment of a high-rise building is treated as one component when the planner makes a rough

construction schedule. However, during the installation, the component needs to be divided into three parts due to the

limited lifting capacity of the cranes (construction method constraint). The 4D modelers should consider this

requirement and the construction method if the model is intended to be used for supporting the construction tasks in

the later construction stage.

According to Hartmann et al. (2008), previous experiences of applying 3D/4D models across different project phases

showed that a large amount of investments into the application of 3D/4D models in later stages in the project

lifecycles could have been avoided if the modelers and project managers had created earlier models with the

intention for later use on the project. However, the 3D models used for different application purposes by various

stakeholders require different levels of detail. The current 4D models for project planning are not detailed enough for

representing physical tasks during construction. For example, Fig. 4 shows that the coordinates for two prisms and

two tags are needed for installation control, however, the 3D as-designed model for 4D planning commonly does not

contain the information of these reference points needed for controlling of position and orientation during

installation.

FIG. 3: Decomposing the model according to construction method

FIG. 4: Design model VS. Production model

During the early stage like bidding stage, it is very difficult for 4D modelers to foresee the above mentioned data as

needed for the later applications, because construction is dynamic and fragmented in nature and different parties have

their own priorities and interests. The 4D system needs to provide interface to input various types of information

by different stakeholders in different stages. One aspect of the important information for handling physical tasks is

related to the control points (fiducials or marks) demonstrated in Fig. 4, which should be aggregated into the design

models associated with the later stages of construction, i.e., the detailing, manufacturing and installation. From the

181


perspective of 4D system development, the system designer should consider how to represent and organize the

building components into objects by using object oriented programming languages. The composite pattern is suitable

to deal with this design problem. Because this pattern ignores the difference between compositions of objects and

individual objects. If this pattern is used to design the structure of objects that represent the building components in

Fig. 3, a 4D modeler can use the same function to manipulate the composite component as well as the three sub-

components.

5. REGISTRATION AND TRANSFORMATION

A variety of data need to be communicated between the system that is responsible for as-built data collection and the

4D system: the geometric data, the registration data, the attribute data indentifying the type of as-built data, and the

time of measurement etc. In order to realize an extensible framework that supports multiple data collection devices,

we designed a two level communication protocol, in which the common data that is identical for all kinds of data

collection devices i.e., the time of measurement, the position and orientation of the device’s viewpoint, and the

identifier of the data type etc. is processed at the first level. Then we used the identifier to indicate the type of as-

built object created in the 4D software system, which in turn would communicate with the data collection system to

receive the data at the second level.

After the as-built data are transferred into the 4D system, it should be registered in the 3D graphic engine before

being integrated with as-designed models. There are a variety of registration processes and methods for different data

collection technologies. The registration of a point is relatively simpler than the laser scanner generated point cloud.

For point registration, it just needs some geometric transformation between a total station’s local coordinate system

and the as-designed coordinate system. But for the raw laser scanned point cloud, it contains many sources of noise,

each requiring a filter for removal. Furthermore, at a single position, a laser scanner can only capture points on an

object in its line of sight. As a result, the occluded parts of the object need to be scanned from other directions. This

requires a process of aligning several scans into a more comprehensive data set (Kiziltas et al. 2008). The diversity of

the registration requires an extensible application framework that allows the registration of various types of data

without manipulating the complex 3D graphic engine.

There are four types of established methods for registration of collected 3D site data (Akinci et al. 2006). The most

important requirement for registration of the site points data is to capture the total station’s position. Considering a

practical construction project, the site usually has fixed markers, also known as fiducials, the position of which are

known for the purposes of site surveying and layout. Those markers are also recognized in the 4D designed model

and are used to aid the registration. If the total station is placed at a known position on construction site, it’s easy to

align the real world coordinate system and the as-designed model coordinate system during registration. However,

when the total station’s position is unknown, its coordinates can be calculated as follows:

(1)

where , and are the coordinates of the point , which are known in advance from the 4D model, and

is the distance between the point and the total station, which can be readily measured by the total station.

When the position of the total station is registered, the surveyed control points on a building component can be

transformed to the as-designed coordinate system. The total station should measure at least three marked points on a

building component in one surveying cycle. One of these points is used to calculate the translation transformation,

which yields a vector: The designed coordinates of the three points denoted

by are known in advance, and the as-built points are surveyed by the total station at a

particular moment of time denoted by , which are then adjusted by the transformation

vector , yielding three new points denoted by . Then the rotation matrix

can be calculated as follows:

(2)

182


is a matrix. Thus, applying the vector and rotation matrix to the as-designed 3D model of the

building component will create a new 3D model to represent the as-built position and orientation of the component

being installed at that time. The transformation vector and rotation matrix can be combined in a transformation

matrix. The concept of this integration and visualization on a case study for installation positioning is demonstrated

in the following implementation section.

6. PROTOTYPING AND IMPLEMENTATION OF THE FRAMEWORK

To validate the feasibility of the framework, a prototype system was developed based on an in-house 4D system

named as 4D graphics for construction and site utilization (acronym 4D-GCPSU) Version 2009. Readers may refer

to Chau et al. (2005) and Wang et al. (2004) for detailed information of its earlier versions. In this prototype system,

the as-designed model is integrated with the as-built data collected by the total station to visualize the position and

orientation deviation of the building component being installed.

The first step is to link the production unit with a position-tracking system which will control the total station to

survey the marked points on a construction unit automatically. To do this, the user selects the construction unit from

the visualization screen or from the WBS tree structure via mouse clicks (see the GUI steps tagged with 1 and 2 in

Fig. 5). The selected production unit is highlighted with a bounding box and also shown in a small window at the

lower left corner. Then by clicking the tracking button (see the GUI step tagged with 3 in Fig. 5), a corresponding

entity will be created (see the GUI step tagged with 4 in Fig. 5) to control the visualization of the as-built position

and orientation of the construction unit in the 3D graphic engine during the installation-tracking process.

FIG. 5: Graphical user interface of the prototype system

FIG. 6: Current practice on positioning the steel columns

183


The case study in Fig. 5 is based on construction of a skyscraper of 103 storeys. It is a composite structure,

comprising a reinforced concrete core tube integrated with the perimeter diagrid frame to provide overall structural

stability. The perimeter diagrid frame has doubly curved elevations in its design such that it is a challenge to

accurately determine the position and orientation of the steel segment being installed. In the current practices, the

contractor uses a total station to measure the geometric center of a diagrid frame segment shown in Fig. 6a.

However, because at least three points are needed to determine the orientation of a solid object, only surveying one

or two points may lead to some orientation errors which cannot be easily detected in the current practice. For

example, in Fig. 6b only measuring point o cannot detect the triangle’s rotation about this point. Because this kind of

rotation will not change the coordinates of the point o. Similarly, only measuring two points (o and p in Fig. 6b)

cannot fix the triangle’s rotation about the line connecting the two points in 3D space.

Furthermore, in current practice the site crew needs to search the designed coordinates during installation

positioning. Significant time in a crew’s shift is actually spent in obtaining and understanding design information. It

is clear that more convenient, timely access to this information would considerably benefits task performance (Wang

and Dunston 2006). In the implementation of the framework, we propose the integration of the designed coordinates

of control points (Fig. 7a) with the real-time data of control point positions collected by a total station in a 4D system

for visualization, which improves the current error-prone approaches and also minimizes the crew’s time spent in

shift between checking design and adjusting installation (Fig. 2).

FIG. 7: The integrated automatic positioning process during installation

FIG. 8: The visualization of the installation process and the orientation error

184


In this framework, two control points are marked on the steel components using reflective tapes in addition to the

two prisms fixed on the geometric center points, one point for determining position and the other three points for

determining orientation (Fig. 7a). The coordinates of these control points as per design for each construction unit are

inputted into the 4D model before installation activities start. So the site crew doesn’t need to search the designed

coordinates during installation. When the installation activity starts, the control points are tracked in real-time by the

total station which will automatically survey each point in one cycle (Fig. 7b). When one surveying cycle is finished,

the coordinates of these points will be transferred and registered into the 4D system. After finishing the registration,

the position and orientation deviation between the as-designed 3D model and as-built component can be calculated,

which yields a transformation matrix including translation and rotation. Applying the transformation matrix to the

3D as-designed model generates a new 3D model to visualize the real position and orientation of the component

being installed, which is rendered in a different colour: the white colour model represents the as-designed

component, and the gray colour model represents the as-built position and orientation of the components being

installed at that time in Fig. 7c.

In Fig. 8a, six snapshots of the integrated visualization of the lifting process are presented in the form of filmstrip to

show the sequence by which the component is gradually approaching its designed position and orientation. When it

is exactly positioned on site the as-designed and as-built models will converge as shown in the last snapshot. Aided

by integration and visualization of as-built and as-designed data, the site crew could easily detect any derivation and

also can get real-time feedback from any installation adjustment operation. This will then further guide the crew on

how to operate the hydraulic jacks or cranes. Fig. 8b zooms in one snapshot which demonstrates a rotation error

around the geometric central point. In the current practice, by surveying the central point alone this error cannot be

determined. This proposed approach to construction-surveying integration and 4D visualization potentially enhances

the efficiency and quality of the installation of steel components.


As-designed and as-built model integration and visualization improve current construction practices in regard to the

tasks of positioning control, guidance of installation, and quality inspection etc. This paper has presented a

framework to integrate and visualize the as-designed and as-built models in a 4D CAD system which extends the

current 4D technology from supporting management tasks to guiding physical tasks. The critical issues on

developing the framework are discussed; the feasibility of the framework is demonstrated with a case study of

structural steel component installation in high-rise building construction. Note that Milgram and Colquhoun (1999)

defined Mixed Reality as “including the full range of combinations of virtual and real environment.” In this case, we

applied the points data collected on site to visualize the position and orientation of as-designed CAD object which

can be regarded as being close to the virtual environment. Thus, the application presented here can be suitably

categorized into augmented virtuality (AV).

The framework can be further extended to accommodate three promising types of data collected from construction

sites, namely, reference points on a production unit being installed, point clouds for representing the actual building

product, and the image of actual building product taken by cameras. The other two types of data will be addressed in

the authors’ ongoing research, aimed to study their feasibility on supporting construction physical tasks, and to

validate the extensibility of the framework.

8. ACHNOWLEDGEMENTS

The research presented in this paper was substantially funded by a National Key Technology R&D Program in the

11th Five Year Plan of China (No. 2007BAF23B02) and a Hong Kong Polytechnic University Niche Area Research

Grant (A/C No. BB89). The writers also thank the West Tower Contractor Guangzhou Construction Group and

China State Construction Engineering Corp. for cooperation and assistance in conducting the research.

185


9. REFERENCES

Akinci, B., Boukamp, F., Gordon, C., Huber, D., Lyons, C., and Park, K. (2006). "A formalism for utilization of

sensor systems and integrated project models for active construction quality control." Automation in

Construction, 15(2), 124-138.

Barrett, P. (2000). "Construction management pull for 4D CAD." Proceeding, Construction Congress Vi, ASCE,

Reston, Va., 977-983.

Bernold, L. E. (2002). "Spatial integration in construction." Journal of Construction Engineering and Management-

Asce, 128(5), 400-408.

Behzadan, A. H., and Kamat, V. R. (2007). "Georeferenced registration of construction graphics in mobile outdoor

augmented reality." Journal of Computing in Civil Engineering, 21(4), 247-258.

Chau, K. W., Anson, M., and Zhang, J. P. (2005). "4D dynamic construction management and visualization

software: 1. Development." Automation in Construction, 14(4), 512-524.

Ergen, E., Akinci, B., and Sacks, R. (2007). "Tracking and locating components in a precast storage yard utilizing

radio frequency identification technology and GPS." Automation in Construction, 16(3), 354-367.

Haque, M. E., and Mishra, R. (2007). "5D virtual constructions: Designer/constructor's perspective." Proceedings of

10th International Conference on Computer and Information Technology (Iccit 2007), 134-137

Hartmann, T., and Fischer, M. (2007). "Supporting the constructability review with 3D/4D models." Building

Research and Information, 35(1), 70-80.

Hartmann, T., Gao, J., and Fischer, M. (2008). "Areas of application for 3D and 4D models on construction

projects." Journal of Construction Engineering and Management-Asce, 134(10), 776-785.

Katz, I., Saidi, K., and Lytle, A. (2008). "The role of camera networks in construction automation." 25th

International Symposium on Automation and Robotics in Construction - Isarc-2008, 324-329.

Kiziltas, S., Akinci, B., Ergen, E., Tang, P., and Gordon, C. (2008). "Technological assessment and process

implications of field data capture technologies for construction and facility/infrastructure management."

Journal of Information Technology in Construction 13, 134-154.

Liston, K. M., Fischer, M., and Kunz, J. (1998). "4D annotator: A visual decision support tool for construction

planners." Computing in Civil Engineering, 330-341.

Ma, Z. Y., Shen, Q. P., and Zhang, J. P. (2005). "Application of 4D for dynamic site layout and management of

construction projects." Automation in Construction, 14(3), 369-381.

Milgram, P., and Jr., H. C. (1999). "A taxonomy of real and virtual world display integration." Mixed Reality:

Merging Real and Virtual Worlds, Ohmsha Ltd, Heidelberg, Germany, 5-30.

Shin, D. H., and Dunston, P. S. (2008). "Identification of application areas for Augmented Reality in industrial

construction based on technology suitability." Automation in Construction, 17(7), 882-894.

Song, J., Haas, C. T., Caldas, C., Ergen, E., and Akinci, B. (2006). "Automating the task of tracking the delivery and

receipt of fabricated pipe spools in industrial projects." Automation in Construction, 15(2), 166-177.

Wang, H. J., Zhang, J. P., Chau, K. W., and Anson, M. (2004). "4D dynamic management for construction planning

and resource utilization." Automation in Construction, 13(5), 575-589.

Wang, X. Y., and Dunston, P. S. (2006). "Compatibility issues in augmented reality systems for AEC: An

experimental prototype study." Automation in Construction, 15(3), 314-326.

186


AUGMENTING SITE PHOTOS WITH 3D AS-BUILT TUNNEL MODELS FOR CONSTRUCTION PROGRESS VISUALIZATION

Ming Fung SIU, BEng,

Dept. of Civil and Structural Engineering

Hong Kong Polytechnic University;

[email protected]

Ming LU, Associate Professor,

Dept. of Civil and Structural Engineering

Hong Kong Polytechnic University;

[email protected]

ABSTRACT: This paper describes an Augmented Reality (AR) application by superimposing computer generated 3-

dimensional graphics over real world photos. Current practice in construction uses separate drawings to represent

the as-designed and as-built structural components, which remains a less effective means of communication.

Applying AR technology has been proposed to remedy this situation but it has yet to exert broad impact on

construction management. The reason is that there exists no accurate, simple and cost effective approach to

implementing this technology. New analytical methods resulting from recent research greatly reduce the effort and

complexity in applying AR. This paper presents a case study of visualizing micro-tunneling construction progress by

applying a new analytical method for augmenting site photos with 3D building product models and contrasts the new

method against "virtual camera" function provided by commercial software. The analytical approach is found to be

accurate and easy-to-apply. This study reveals the possibility of developing an automated solution based on the

analytical method of augmenting a series of time-lapse site photos with 3D as-built models, potentially benefiting

practitioners in visualizing construction progress of subsurface pipeline infrastructure.

KEYWORDS: Augmented reality, Site photo, Construction progress, Visualization, Automation.

1. INTRODUCTION

Graphical simulation by augmented reality (AR) technology provides a useful tool for evaluating the building

product against its design and investigating the constructability of construction activities during planning and

construction stages. AR technology has made headways in recent years, driving the acceptance of AR by researchers

and practitioners. For instance, comprehensive AR applications in construction were evaluated by Shin and Dunston

(2008). Several experiments were carried out for illustrating the feasibility of applying AR such as superimposing a

CAD model of an excavator over a real site background in operations simulation (Behzadan and Kamat 2007).

In previous attempts of applying AR technology, the tracking system relied on GPS and three-degree-of-freedom

orientation tracker to determine the position and orientation of camera’s viewpoint. However, the system has yet to

provide speed and accuracy for achieving real-time tracking (Kensek and Noble 2000). In the development of AR

technology, overlapping view frustums of both virtual and real photos results in augmented photos. Generally,

manual manipulation is needed in fine-tuning the position of the model; as a result, the process is time consuming

and the AR modeling effort is hard to repeat. For instance, Kim and Kano (2008) conducted an experiment to

illustrate the difference between the as-built products and as-design drawings by superimposing actual construction

photos and virtual reality (VR) images. However, manual adjustment was needed to align and superimpose the

virtual and actual scenes. Thus, it raises the present research interest in employing AR by using a camera only and

processing the superimposition analytically and automatically.

An analytical approach to augmenting site photos with the as-build bored pile model has been proposed based on the

photogrammetry theories by Dai and Lu (2009). It has not been attempted yet to visualize a completely underground

feature and compare the new analytical approach with the manual approach. To enable engineers or clients to have

187


an integral view of the actual construction site situation and process, dynamic visualization of a pipeline tunnel under

construction will also be presented in this paper. In short, augmented construction photos will be valuable in

documentation, progress controlling and construction monitoring in practice. This paper focuses on the comparison

between the new analytical approach and the manual approach. It also discusses the potential of developing an

automated solution based on the analytical technique and the value added to practice with a case study of visualizing

micro-tunneling progress.

2. RESEARCH PROBLEM DEFINITION

In the practical micro-tunneling site, a site location plan (2D CAD drawing) was generally employed for presenting

the location, layout and alignment of the structure. As for the project, a method statement indicated that "a pre-

condition survey with sufficient site photos showing the ground condition, existing vegetation and plants etc. is

required. " However, there was no effective way to express the construction site situation together with the proposed

tunnel design and to illustrate the progress of on-going pipe jacking operations. In this case, only laser was used in

controlling the alignment of the tunnel. The resulting survey records on the alignment control for pipe jacking were

submitted to the client. Piles of data sheets made the recording and communication difficult and confusing. Hence, it

is worthy of investigating the potential in generating augmented photos to visualize the as-designed underground

structure alongside the as-built progress of the tunneling project in dynamic views, and in a cost effective way.

The aim of this study is to verify the potential for visualizing 3D as-designed or as-built models with real site photos

by applying an analytical photo-augmentation approach. The research is also intended to generate a series of

dynamic views of tunneling construction by using as-built data analogous to time-lapse video keeping. In conducting

the case study, the new analytical technique and the traditional approach are contrasted, and the potential of

automating the newly developed analytical approach along with site constraints and limitations are evaluated.

3. ANALYTICAL METHOD VS MANUAL MATHOD

The new analytical method is based on photogrammetric theory (Dai and Lu 2009). Photogrammetry is a science or

art of obtaining reliable measurements by means of photographs (Thomson, et al. 1966). The method is briefly

introduced as follows: by defining two known points, a camera’s perspective location O ( ) and the

object’s focus location ( ), a normal vector N is introduced by linking the two points as illustrated in

Fig. 1a. It is a vector perpendicular to the image plane of the camera, which extends from the perspective center of

the camera to the focal point of the photo. Three orientation angles regarding the orientation of camera can be

computed by using the normal vector. These three angles control the orientation of the camera. Fig. 1a and Fig. 1b

show clearly the orientation of the perspective center of the camera with respect to the object coordinate system.

FIG. 1a: Orientation angles FIG. 1b: Orientation angle s

188


With the Normal Vector N, the three orientation angles ( ) defining the orientation of camera are computed by

Eq. (2) and Eq. (3).

(1)

(2)

(3)

Regarding the rotation of in the Fig. 1b, with s measured from the positive y-axis to the

photogrammetric nadir point. By defining to be , . This implies the orientation of the camera in taking

picture is always in the "landscape" mode.

FIG. 2a: Original orientation of the camera FIG. 2b: Rotate about axis

FIG. 2c: Rotate about axis FIG. 2d: Rotate about

In taking photos, the camera is only allowed to rotate and angles about axis and axis respectively, as

shown in Fig. 2a and Fig. 2b. Rotation about the normal vector N (Fig. 2d) is prohibited to ensure the camera taking

photos in "landscape" mode.

With the three orientation angles, the relationship between nth target points in the object space to the corresponding

point in the image plane can be established by the Collinearity equation as shown in Eq. (4).

189


That is,

(4)

where

(5)

and

(6)

Another form of Collinearity equations with the scaling factor being eliminated is given in Eq. (7) and Eq. (8).

In these equations, c is the focal length of the camera, ( ) is the coordinates in the image coordinate system and

( ) is the coordinates in the object coordinate system. By substituting a known point’s coordinates in the

object space into the equation, the corresponded image coordinates can be calculated.

(7)

(8)

That is, ( ) = (9)

Eq. (9) implies that the image coordinate is a function of object coordinate. Thereby, through establishing a

mathematical regular grid platform the two photos can be superimposed analytically and precisely. The grid can be

achieved by using either the CCD/CMOS size parameters or the "extra point" method, as are described below.

The image coordinate plane (or termed as the photo coordinate plane) in the camera lies on the Charge-Couple

Device (CCD) or the Complementary Metal Oxide Semiconductor (CMOS) with different camera configurations.

They are image sensors with different technologies for capturing images digitally (DALSA Corporation, 2009)

while their mechanisms are similar in capturing an image. An image is projected through a lens onto the photoactive

region of CCD or CMOS, then, the photo being captured is digitized and stored in some form of memory processed

by the controlling circuit. The size of the image coordinate system is defined by the size of CCD or CMOS. Hence, a

mathematical regular grid platform can be established from the dimensional parameters of CCD/CMOS for

superimposition.

The CCD/CMOS technique proposed provides a means to fit a real photo into the definite size of the frame. The size

of the frame is controlled by the dimensions of CCD or CMOS, with the focal point being the centre of the photo.

The image coordinates of each point in the real photo can be evaluated on this grid. By using the image coordinates

190


of one more point from the object coordinate system, in addition to the focal point in the virtual photo, a virtual

photo can be superimposed with the associated actual photo precisely.

The "extra point" method is used if the size of CCD/CMOS is not known. One extra point is needed in the actual

photo in order to set up the grid together with the focus point, which is similar to the virtual photo processing being

discussed. Then the two photos can be superimposed by using the grid precisely.

In regard to the manual method, some commercial software, such as Autodesk 3ds Max® Free 30-day trail software

(Autodesk Inc. 2009), a photo being captured by a real camera can be inserted as environmental map (specifically,

camera map) to render a background scene for a virtual camera. Then, augmented photos result in this way.

The use of the "virtual camera" concept is common in commercial of 3D graphics. Theoretically, if the focal length

of the lens of the virtual camera and the real camera are the same, given the identical position and orientation of the

camera, the scale of the images in the photo is fixed. However, the dimension and aspect ratio of photos vary with

cameras of different models and brands. Resizing, cropping and rescaling need to be done manually before the

photos can be superimposed correctly. Notwithstanding manual operation is feasible, it is not a quick and convenient

approach in applying the proposed AR technology in construction. Automation by the use of analytical method will

of benefit in terms of accuracy, time and cost.

In contrast with the traditional superimposing technique, the newly developed analytical approach accurately

calculates the image coordinates in both real and virtual photos. These virtual vs. actual photos are seamlessly

mapped together on the coordinate platform, thus outperforming the manual method in terms of accuracy and

automation potential.

4. CASE STUDY

A real case study was conducted to compare the analytical approach and the manual approach. It was micro-

tunneling construction by jacking concrete sleeve pipes across the So Kwun Wat Nullah in Hong Kong, consisting

of pit construction and installation of 1200mm internal diameter precast concrete sleeve pipes by pipe jacking

method as shown in Fig. 3. Two parallel micro-tunnels were planned to be constructed. When the case was

conducted, the construction of the micro-tunnel for electrical cable had been completed and the other was in the

planning stage.

FIG. 3: Jacking pit and concrete sleeve pipe

4.1 Site Constraints

Before conducting real site experiment, site investigation was carried out to determine the possible locations of

setting up the camera station and fixing the focus point in the context of site constraints. Through the investigation,

it was found that identifying a location with a view showing the whole construction site and covering the whole

tunnel was difficult. The reason was the construction site was relatively large. The focus point could not be chosen

at the centre of the Nullah since the coordinates of the focus point could not be determined by surveying

instruments. Thus, the location of the focus point should be near the centre of the construction site where the

coordinates were readily available.

191


The camera location was carefully selected to provide a clear view to show features on the surface and the

underground tunnel being built. The main site constraints in this study were the vegetation obstructing the view

while taking photos, inaccessible location within confines of private residences as well as too high or too far

locations to capture construction site photos in sufficient details.

4.2 On-Site Work

After the locations of the camera’s perspective location and the focus point had been evaluated, the photo used for

3D model augmentation was shown in Fig. 4, with the object focus location being indicated at the centre of the auto-

focus point of the camera. The model of the camera was Canon EOS 400D. The camera location and the focus point

were determined. On the site, there were known surveyed control points used to define positions in three dimensions

(Easting, Northing and Zenith). Those points played an important role in planning and design stages, execution of

work and as well as construction monitoring purpose. Well-practiced surveying techniques were used in determining

the coordinates of the camera’s location by using surveying instruments, including Total Station, reflector and

measuring tape. The coordinates of the camera and the focus point surveyed were (817537.3, 825641.0, +10.85) and

(817574.5, 825675.1, +3.10) respectively.

FIG. 4: Photo being used in augmentation

4.3 Off-site work

In the virtual reality environment of 3ds Max®, 3D models can be built by using the data from as-design drawings.

All the coordinates surveyed are respected in the World Geodetic System. It is difficult to input those coordinates to

the 3ds Max® directly. Transforming the global coordinate system to the local coordinate system by using 2D

planar motion transformation of points (Manual of photogrammetry, 5th ed., 143-152) is more convenient.

In the model, the origin is selected at the centre of the cross section of the micro-tunnel for electrical cable. The

transformed coordinate of the camera’s perspective location O ( ) and object focus location with

coordinate ( ) are (26.640, 238.713, 15.003) m and (2.560, 194.200, 7.250) m respectively. With the

Normal Vector N, the three orientation angles ( ) have been computed as (-151.6 o, 81.29 o, 180.00o).

(10)

(11)

With the transformed geometric data, the 3D tunnel model was built within 3ds Max® directly. The resulting model

is shown in the Fig. 5. With the precise position and orientation of the virtual camera calculated by the analytical

method, a virtual camera was created and. Fig. 6 shows the position of virtual camera and a virtual photo being

generated in the virtual environment.

Focus point

192


FIG. 5: As-designed 3D tunneling model FIG. 6: Virtual camera positioning

In the merging process, at least two sets of coordinates in the image coordinate system ( ) in respect of the real

and virtual photos are needed in superimposition by using the "extra point" approach. Thus, by computing at least

one extra point’s coordinates in the object space, the corresponding coordinates in the image plane could be

evaluated and the grid platform could be generated with the focal point. The actual photo and the virtual photo could

then be merged into an augmented photo by common graphic software, such as Photoshop® Free 30-day trail

software (Adobe Systems Inc. 2009).

In this case study, the CCD/CMOS approach has been employed since it was not straightforward to obtain one extra

point’s position in the real photo. The Canon EOS 400D camera used is a single lens reflex digital camera with a

CMOS APS-C sensor. The dimensions of the sensor are 22.2 x 14.8mm. Thus it was used in setting up the grid in

the image coordinate plane.

By means of inputting one more point from object coordinate system in addition to the focal point, the image of real

and virtual photos could be merged analytically. The underlying computation of rotational matrix Eq. (12) is

introduced in Section 3.

(12)

The point used was the end of the micro-tunnel for electrical cable, the other end of which was the origin as

mentioned before. With the focal length f equal to 18mm, the transformation from the object coordinate (0, 220, 0)

to image coordinate using Collinearity equations are shown in the Eq. (13) and Eq. (14). By plugging in the value (0,

220, 0) to ( ), the image coordinate of the target point was calculated as (8.416180, -6.035576) with

respect to the focal point (0, 0) at the center of the imaging plane. Then merging the two photos was processed on

the grid mapping provided by the photo-editing software Photoshop®. The resulting photo is consistent to that

obtained by the manual approach as shown in Fig. 7a.

(13)

(14)

It is recognized that analytical method is an accurate method since the photo-augmenting process is based on

computing with image coordinates. Accuracy can be further improved by calculating more extra target points and

applying least square adjustment technique to adjust the positions and scale of the augmented photo. The major

advantage is that the whole process is computation based, which means automation can be achieved by developing

computer software, without any manual adjustment to superimpose real and virtual photos. The main limitation of

this approach is the image coordinate system must be determined before photo superimposing and augmentation.

The automation cannot be done without the established gird platform.

193


Further modification was made in highlighting the underground feature of the micro-tunnel by adding a jacking pit

and a receiving pit as shown in Fig. 7b. The jacking pit and receiving pit were constructed by excavation with sheet

pile shoring systems. By adding those pits in the augmented photos, the as-designed underground tunnels could be

clearly seen submerge below the ground level indicated.

FIG. 7a: Augmented Photo FIG. 7b: Augmented Photo with pits

Furthermore, construction site progress can be shown by inserting time dimension to the augmented photos. In

practice, the installed length of the tunnel could be measured by counting the number of pipe segments being jacked.

As-built construction progress could be visualized easily by using construction process records with respect to the

cumulative tunnel length. The total duration of the project was 66 days. The total length of the tunnel was

approximately 220m. Augmented photos representing actual site situation on particular project days are shown in

Fig. 8.

Day 11, length of tunnel is 130m Day 22, length of tunnel is 170m Day 33, length of tunnel is 183m

Day 44, length of tunnel is 198m Day 55, length of tunnel is 203m Day 66, length of tunnel is 220m

FIG. 8: Construction progress visualization

Underground tunnel

Ground level

194


5. CONCLUSIONS

This paper focuses on the comparison between the analytical technique resulting from recent research and the

manual method for augmenting photos with 3D as-built tunnel models. A real case study was presented to discuss

the site constraints and limitations during site experiments on analytically augmenting site photos with 3D tunnel

models. Then, by computing the camera’s position and orientation, the perspective views of the virtual camera and

the real camera have been aligned and the resulting photos superimposed on a regular grid platform. The augmented

photo-based construction site progress visualization was demonstrated also. Contrasting the manual approach, it is

found that the analytical approach in applying photo augmentation is more accurate and potentially programmable

(automation procedure shown in Fig. 9). This new approach lends a high-quality visualization method to benefit

practical applications in construction.

FIG. 9: Conceptual design of automation process

6. ACKNOWLEDGEMENTS

The writers would like to express their sincere thanks to Mr. K.C. Ng, Regional Manager-North, of CLP Power

Hong Kong Limited, Mr. Sam C. K. Shum of The Hong Kong and China Gas Company Limited, Mr. Joseph H. Y.

Kong and Mr. Lotus C.W. Au Yeung of Kum Shing (K.F.) Construction Company Limited, Mr. Eric K.I. Lo and

Mr. William, W.Y. Leung of Black & Veatch Hong Kong Limited, for providing micro-tunneling construction site

access and first-hand data. Thanks are also due to Mr. F. Dai for his guidance, and thought-provoking comments.

This presented research was substantially funded by Hong Kong Research Grants Council (RGC) through Project

PolyU 5245/08E.

7. REFERENCE

Adobe Systems Inc. (2009). "Featured product downloads. " Available via < http://www.adobe.com/downloads/>

[access: 01-Jun-09]

Autodesk Inc. (2009). "3ds Max®: Production-proven 3D modeling, animation, and rendering solution for games,

film, television, and digital publishing." Available via <http://usa.autodesk.com/adsk/servlet/index?id=

5659302&siteID=123112> [access: 01-Jun-09]

Behzadan A. H. and Kamat V. R. (2007). "Georeferenced Registration of Construction Graphics in Mobile Outdoor

Augmented Reality. " Journal of computing in civil engineering, 21(4), 247-258.

Dai F. and Lu M. (2009). "Analytical approach to augmenting site photos with 3D as-built bored pile models. " In

proceeding of the 2009 Winter Simulation Conference (M. D. Rossetti, R. R. Hill, B. Johansoon, A. Dunkin

and R. G. Ingalls, editors).

Output Computer Automation Process Data Input

Coordinate of camera’s

perspective location

O ( )

Coordinate of focus point

( )

Extra point in virtual photo

+ Extra point in real photo

or CCD/CMOS size

Calculation of

orientation angles

Setting up grid

platform

Superimposition

of virtual and real

photos

Augmented Photo

195


DALSA Corporation. (2009). "CCD vs. CMOS. " <http://www.dalsa.com/corp/markets/CCD_vs_CMOS.aspx>

[access: 28-Jun-09].

Kensek K, Noble D., Schiler M. and Tripathi A. (2000). "Augmented Reality: An Application for Architecture. "

Proceedings of the Eighth International Conference on Computing in Civil and Building Engineering, 294-

301.

Kim H. and Kano N. (2008). "Comparison of construction photograph and VR image in construction progress. "

Automation in Construction, Vol. 17, No. 2, 137-143.

Matossian M. (2004). "3ds max 6 for Windows. " Berkeley, Calif.:, PeachPit Press. 375-459.

McGlone J. C., Mikhail E. M., Bethel J. and Mullen R. (2004). "Manual of photogrammetry. " 5th ed. Bethesda,

Md.: American Society for Photogrammetry and Remote Sensing. 1-2, 143-152

Murdock K. (2008) ."3ds Max 2008 bible. " Indianapolis, Ind.: Wiley Pub., Inc.. 35-324

Sin D. H. and Dunston P. S. (2008). "Identification of application areas for Augmented Reality in industrial

construction based on technology suitability. " Automation in Construction, Vol. 17, No. 7, 882-894

Wang X. Y. and Dunston P. S. (2005). "Real Time Polygonal Data Integration of CAD/Augmented Reality in

Architectural Design Visualization." Proceedings of the 2005 ASCE International Conference on Computing

in Civil Engineering, 1-8.

196


AUTOMATIC GENERATION OF TIME LOCATION PLAN IN ROAD

CONSTRUCTION PROJECTS

Raj Kapur, Shah, PhD Candidate:

CCIR, School of Science and Technology, University of Teesside, Middlesbrough, TS1 3BA, UK,

Email: [email protected]

Nashwan Dawood, Professor:

CCIR, School of Science and Technology, University of Teesside, Middlesbrough, TS1 3BA, UK,

Email: [email protected]

ABSTRACT: Because of unique characteristics of earthwork activities in road projects, construction managers

and planners require innovative techniques to assist them in producing accurate planning tasks such as efficient

resource allocation and costing of activities. The research study introduced a framework of an innovative model

dubbed “Virtual Construction Model (VCM)”. The paper focuses on automatic generation of a time location

plan and conflict identification system for earthwork activities in a road project. The framework is designed by

integrating road design data, quantities of cutting and filling sections, variable productivity data, algorithms for

4D terrain modelling, and a time location plan generator. The model is validated with a real life case study of a

road project and it was found to be beneficial in generating the 4D terrain surfaces of progress and a time

location plan with more accurate information of location and quantities in the earthwork operations. The VCM

has potential to assist project planners and construction managers in producing efficient construction

scheduling and resource planning.

KEYWORDS: Earthwork, Productivity, Sensitivity Analysis, Time Location Plan, Virtual Construction Model

1. INTRODUCTION

The planning, scheduling, and controlling system adopted by project planners and construction managers

determines the success of any construction projects. Construction managers and project planners of linear

construction projects such as roads, railways, and pipelines require advanced project planning and scheduling

tools to control budget, schedule and resource allocation so that project goals could be achieved on time and on

budget. The effective application of planning and scheduling techniques such as CPM and PERT is limited

because road construction activities are fundamentally different to building construction projects (Hamerlink

and Yamin, 2000).

In a large-scale project, a visual representation of the construction schedule can be extended to monitoring not

only the construction progress, but also all the auxiliary activities, including onsite plant and equipment (Adjei-

Kumi et al, 1996). McKinney et al (1998) demonstrated the capability of 4D-CAD models to identify the

construction problems prior to their actual occurrence. The failure to provide the information of spatial aspects

of a construction project by traditional techniques such as Bar Charts and the Critical Path Method (CPM) have

motivated the research effort to incorporate visualisation techniques into project scheduling and progress

control (Koo and Fischer, 2000). Zhang et al (2000) further developed a 3D visualization model with schedule

data at the level of construction components. Dawood et al (2002) developed an integrated database to act as an

information resource base for 4D/VR construction process simulation and it was applied to a building project.

Furthermore, several research efforts carried out in the visualisation of the construction process applied to

building construction projects, but there have been limited research studies in the area of infrastructure

construction projects. For example, Liapi (2003) focused on the use of visualisation during construction of

highway projects to facilitate collaborative decision making on construction scheduling and traffic planning,

however, the visualisation of the construction schedule for the intermediate stages of the construction process

was neglected. Castro and Dawood (2005) developed the “RoadSim” simulator based on the site knowledge-

based simulation system. It is applicable to develop a master construction schedule in a road project based on

simulated productivity of road building activities and available resources with different sets of equipment and

site working conditions. Kang et al (2008) suggested an approach to simulate 4D models for the movement of

197


earthwork activity for the intermediate stage of the construction process in civil engineering projects using

morphing techniques and realisation of construction progress in graphical images. The 4D models of earthwork

operation have been produced in 3D CAD model at equal volume and at a fixed production rate of the

earthwork activities at different stages during construction operation and linked with time but the variable

productivity data of equipment and soil characteristics was not considered in the 3D CAD models.

The above research efforts did not address the interface of variable production rate, which depends on available

resources and site conditions for the development of the VCM. The key issue faced in road construction sites is

the variable productivity from one day to another due to the special characteristics of the road construction

industry; such as fluctuation in daily weather conditions, working conditions in open sky, resource

unavailability on time and other unpredictable factors. The study focuses on addressing the above issues by the

development of the VCM. The model will be integrated with the “variable production rate” of earthwork

activity throughout road construction operations. Currently accepted scheduling techniques including CPM,

PERT and Bar Charts are unable to model linear activity more accurately in terms of locations. A linear

scheduling method developed earlier than CPM has the potential to provide significant enhancement, because it

provides location of working activities coupled with the advancement of computer technology. This allows the

project schedulers and construction managers to plan road construction project visually to determine the

controlling activity path (Hamerlink and Yamin, 2000).

Previously research efforts by Johnston (1981 and Garold et al, 2005) in the area of linear scheduling concluded

that the techniques are a useful scheduling tool for progress monitoring in road construction projects during the

planning and execution phases. Previous research studies have considered earthwork activities as a linear

activity (Hamerlink and Yamin, 2000). However, earthwork activities are nonlinear in real practice since the

earthwork quantities vary along a road project from station to station (chainage to chainage) according to

topography of terrain surfaces. To overcome this issue of the earthwork activity, this study presents an

innovative methodology for the development of VCM and a time location plan of the earthwork construction

processes in road construction projects. The model intends to assist in improving the site communications of

road construction planning and scheduling information, and to produce efficient construction scheduling and

resource planning. The remainder of this paper outlines a conceptual framework and details of the prototype for

VCM and generation of a time location plan of the earthwork operations in a road project.

2. FRAMEWORK OF A VIRTUAL CONSTRUCTION MODEL (VCM) The general specification of framework of a prototype of virtual construction model is outlined in figure 1. The

framework integrates the road design data, sectional quantities of cut and fill, productivity models, algorithms

for 4D terrain modelling and a time location plan generator. The model assists in generating visual terrain

surfaces of road progress automatically throughout the earthwork operations. The next section describes in

detail the input, process and output of the VCM.

FIG. 1: Framework specification of a Virtual Construction Model (VCM)

198


2.1 Input

The sectional quantities of cut/fill of earthwork activity along a road section, productivity of the activities and

construction site knowledge base are key components of the framework. The sectional cut/fill quantities are

calculated using road design data including L-sections and X-sections at required intervals of chainage. The

productivity data, which is produced by the “RoadSim” simulator, is used as a key input in the model. The

productivity is calculated using the available resources, equipment sets and site working conditions. This is

incorporated with the model to determine the total duration of the earthwork operations. The soil characteristics

along the working road section, types of available equipment set for a selected activity, haulage distance of soil,

working conditions and all other factors including weather conditions that control productivity has already been

incorporated within the “RoadSim” simulator. Additionally, the model will assist in identifying the possible

location and numbers of site access points. The construction knowledge will assist to select the methods of

construction process for different types of soils, equipment sets for a particular activity and soil characteristics.

The site operational rules allow in establishing the sequential relationships amongst listed activities during the

construction operations. The following section describes and demonstrates the process of the VCM.

2.2 Process

The process of the framework includes four modules: data generation module, visualisation module, cost profile

module and a time location module. Data generation module processes the input data to produce a detailed

schedule and to generate the coordinate data based on the production rate i.e. on the weekly or daily basis in

this study. The visualisation module processes the coordinate data produced by the data generation module, and

converts it into terrain surfaces of the road progress profiles. The cost profile module generates weekly cost

profiles/histograms and the time location module generates a time location plan of the earthwork operations in

road projects. This paper focuses on the development of time location module and conflict identification system

are presented in the following sections.

2.2.1 Generation of Time Location Plan

In this section, a set of algorithms is developed to automate the generation of terrains of earthwork activities at

different stages of the construction process. This is considered the quadratic equations, which determines the

progress height for earthwork activities. A detailed development for the generation of weekly coordinate data

presented by Shah et al (2008) is used for the generation of a time location plan in this research study.

The time location plan is also known as time-distance planning, time-chainage planning and linear scheduling

method (LSM). It enables the creation and display of planning and scheduling information of earthwork

activities in two dimensions: Location in X-axis and Time in Y-axis or vice versa together with the

topographical information of a road project. The slope of activities displayed in time location plan represents

the rate of productivity. The slope of activity provides the early indication of conflicting or overlapping

activities that may occur during the course of activity progress.

An algorithm was designed to identify the start and end location as well as start and end time of earthwork

activities. The developed algorithm determines the location (chainage) along a road section and are broken

down into weekly schedules satisfying the linearity characteristics (start and end locations having equal

production rate) of the earthwork activities. The identified locations and time are summarised and presented in

a table. A linear schedule of the earthwork activities is generated from the tabulated locations and time as

(coordinates of the starting and ending points of weekly earthwork activities) developing a module (macro)

based on using Excel VBA (Visual Basic for Applications) programming.

The generated time location plan provides clearer representation of a construction schedule and enables the

visualisation and analysis of the status of construction activities on a particular location along the road sections.

It also supports in identifying the possible conflicting/overlapping locations along the road section. The detail

of the development of conflict identification system is described in the following sections.

199


2.2.2 Development Conflict Identification System (CIS):

This section focuses on exploring and developing a new methodology in which the VCM can be enhanced to

represent earthwork scheduling and planning information in a time location plan (TLP) and to identify the

possible conflicting points. It is envisaged that TLP will provide more accurate information in terms of location

through integrating with a real site operations and incorporating actual earthwork progress data with the VCM.

In this way, the model enables the integration of real site data of soil profiles and assists to update site

productivity in earthwork operations according to soil characteristics along a road project and scheduling

information is represented in a time location plan. The flow diagram of the conflict identification system (CIS)

is presented in figure 2 and the details of the flow diagram are described below.

The conflict identification system is important to construction managers and project planners in order to

identify the possible overlapping activities in advance so that space conflicts, wastage of resources, idle

equipment and reduction in site productivity can be resolved at planning stage. It is anticipated that conflict

identification system can assist to reduce the remaining difficulties encountered by construction managers when

allocating resources and monitoring the site progress. To resolve the above issue, this research study focuses on

a conflict identification system in a road project including earthwork operations. The functionality of VCM has

been improved by incorporating a new approach, which is useful to identify and determine the overlapping or

conflicting points in terms of location and time along a road section between two activities in earthwork

operations.

FIG. 2: Data flow diagram of Conflicting Identification System (CIS)

200


Location (L)

Time (L)

Slope (Productivity) of earthwork Activity A1 = m1

Slope (productivity) of sub base activity A2 = m2

A (L1, T1)

B (L2, T2)

C (L3, T3)

D (L4, T4)

Conflict point (L, T)

The conflicting identification system (CIS) is expected to assist project planners and construction managers in

identifying and determining the coordinates (location and time) of the conflicting/overlapping points having

different productivity value and passing through different site access points. The location and time of

conflicting/overlapping activities during earthwork operations can also be analysed considering the soil

characteristics along a road section.The system enables project planners and construction managers to take

preventive measures by forecasting the possible conflicting activities in earthwork operations. If planned

productivity varies time progression due to variation in soil characteristics or site access points along the road

projects, there is a chance of overlapping or confliction between activities with different productivity. The

following section describes the detailed derivation of mathematical formula for the identification of conflicts

between two earthwork activities.

2.2.3 Derivation of mathematical formula for conflicting location and time:

FIG. 3: Time Location Plan for earthwork activities A1 and A2

Considering figure 3 for determining the conflicting/overlapping point between activity A1 and A2:

Assume, Line AB represents earthwork activity (A1) which is passing through point A (L1, T1) and B (L2, T2),

and Slope of the line AB is represented by m1 . Similarly, Line CD represents sub base activity (A2) which is

passing through point C (L3, T3) and D (L4, T4), and Slope of the line CD is represented by m2

Slope of line AB (activity A1) = m1 = (T2-T1) / (L2-L1) ……….. (1)

i.e.; Productivity of earthwork activity A1 is expressed in linear metre/week

Similarly, Slope of line CD (activity A2) = m2 = (T4-T3) / (L4-L3) ... …….. (2)

i.e.; Productivity of earthwork activity A2 is expressed in linear metre/week whereas; C1 and C2 are the

intercept of line AB (activity A1) and CD (activity A2)

According to coordinate geometry, Eq. of a straight line passing through point (x, y) and having slope (m) and

intercept c is y = m x + c. Similarly, equation of a straight line AB having

Slope (m1), intercept C1 and passing through point (L1, T1) is expressed in equation 1 below

T1 = m1*L1+C1 ……………. (3)

Equation 3 is derived by algebraic transformation of equation 1;

C1 = T1- m1*L1 ……………… (4)

If the line AB is passing through a point (L, T), the equation of the line AB as below:

T = m1* L + C1 ………………. (5)

Substituting the value of C1 in Eq. 3;

T = m1 *L + T1 - m1*L1;

Or, T = m1 *(L - L1) + T1 …….……….. (6)

201


Similarly, Equation 7 can obtain after deriving, equation of line CD having slope (m2) and passing through (L3,

T4) is; T = m2 *(L-L3) + T3 …………… (7)

Equations 8 and 9 can be obtained by solving above equations 4 and 7 for identification of

conflicting/overlapping location at distance (L) and Time (T)

L = [(m1*L1-m2*L3) + (T3-T1)]/ (m1-m2) ………… (8)

T = [m1* m2 *(L1-L3) + m1*T3 –m2*T1]/ (m1-m2) ………… (9)

Therefore, equation 8 will provide the location where two activities overlap each other and equation 9 will

provide a time when both activities overlap each other at the conflicting location. An illustration of the system is

presented in the following section.

2.2.4 Demonstration of conflict points calculation:

FIG. 4: Two typical work activity having different productivity

Assume, line AB represents earthwork activity (A1) of earthwork at week 1 which is passing through starting

point A (100 m, 1wk) and ending at point B (200 m, 2 wk). similarly, line CD represents sub base activity (A2)

of earthwork at week 2 which is passing through starting point C (75 m, 2wk) and ending at point D (225 m, 3

wk) as shown in figure 4. According to the slope of a straight line; m = y2-y1/x2-x1. Therefore, Slope of the line

AB (activity A1) = m1 = (T2-T1)/ (L2-L1) = (2-1) / (200-100) = 0.01 m/wk,

Similarly, Slope of the line CD (activity A2) = m2 = (T4-T3)/ (L4-L3) = (3-2) / (225-75) = 0.0067 m/wk

Using equation 8, the conflicting point at the location (L) is determined as below;

L = [(m1*L1 - m2*L3) + (T1-T3)]/ (m1-m2) = [(0.01*100 - 0.00667*75) + (1-2)] / (0.01-0.00667) = 450.04 m.

Furthermore, the conflicting/overlapping at time (T) is determined by equation 9. T = [m1* m2 *(L1-L3) +

m1*T3 –m2*T1]/ (m1-m2) = [0.01*.0667(100-75) + 0.01*2- .00667*1]/(0.01-.00667) = 4.5 wks. Therefore, the

activities A1 and A2 will conflict or overlap at point (L, T) = (450 m, 4.5 wks)

3. DEMONSTRATION OF THE VCM WITH A CASE STUDY

3.1 Case study development:

A real life case study involving 1.5 km of road section of lot no. 3 road project in Portugal was selected and

demonstrated the model for earthwork activity of cut to fill or spoil. For this purpose, actual road design

parameters and geometric data of the L - section and the X-section is considered, and the sectional quantity of

earthwork is calculated assuming the typical trapezoidal sections at 25 m intervals along the selected length of

road section. The maximum point of cut/fill section is identified where construction operations start first as per

existing practice and construction site knowledge. Innovative arithmetic algorithms and derived mathematical

equation for the height calculation is designed, developed and validated by the authors The detailed derivation

of the equation and algorithms was discussed and presented in Shah et al (2008).

In this case, progress height is presented by Z- coordinate whereas X direction represents along the road and Y

direction represents along the cross section. The road surface is presented in terms of height in mesh form. The

productivity of the selected activity is the key variable to identify the next surfaces/layers in the construction

progress. The next surface/road profile has been developed based on remaining sectional quantity after progress

of earthwork equivalent to the weekly production rate. The operations repeats for the next economical stretch of

length where the cutting and filling operations take place in order to generate earthwork progress profiles

A =100, 1 Conflicting point

D= 225, 3 C= 75, 2

B = 200, 2

202


automatically of a road section. The economical stretch (balance line) has been identified using the mass haul

diagram rules and it is used in the model.

3.2 Time Location Plan Generated by the VCM: The developed VCM has additional capability to generate a time location plan as a construction planning and

scheduling tool for a road section throughout the earthwork operations as shown in figure 8. The coordinates of

starting and ending location of earthwork activities with corresponding start and end date in week or day is

generated as shown in table 1 using an algorithm and integrating it with the VCM. The data of the table is used

to generate a time location plan as shown in the figures 9 (a) and (b) using VBA programming. The time

location plan is integrated with variable productivity data so that the time location plan can be generated

automatically according to any changes in site conditions, access point, equipment productivity and soil

characteristics throughout the construction operations.

Table 1: coordinate data of start, end location, start, and end date for cut/fill activities S.N. X1 (Start

Station) m

X2 (End

Station) m

Y1 (Start

Date) day

Y2 (End

Date) day

Cut/Fill

w0 125 200 0 1 F

w1 100 200 1 2 F

w2 100 200 2 3 F

w3 100 200 3 4 F

w4 0 200 4 5 F

w5 0 225 5 6 F

w6 0 225 6 7 F

w7 0 225 7 8 F

w8 0 225 8 9 F

w9 0 225 9 10 F

w10 0 225 10 11 F

w11 0 225 11 12 F

w12 0 225 12 13 F

w0 325 425 0 1 C

w1 325 475 1 2 C

w2 300 475 2 3 C

w3 300 475 3 4 C

w4 300 475 4 5 C

w5 275 475 5 6 C

The number of weeks required for a cutting or filling section is represented by weeks such as w1 (filling at day 1

in blue colour), w1 (cutting at day 1in green colour) as shown in index of figures 5 (a) and (b). Figure 5(a)

shows when using two set of filling and one set of cutting equipment and figure 5(b) shows when using one set

of filling and one set of cutting equipment. Similarly, the coordinate of starting and ending point of activity is

also presented in terms of location and time (m, day) as shown in figures 5 (a) and (b) including with

comparisons with existing time location shown on dotted line.

FIG. 5 (a): Snap shot of automatic generated time location plan using two sets of filling and one set of cutting

equipment for earthwork operations.

Filling activity

Cutting activity

Existing time location plan of cut/fill activity

203


The conflicting identification system (CIS) is expected to assist project planners and construction managers in

identifying and determining the coordinates (location and time) of the conflicting/overlapping points having

different productivity value and passing through different site access points. The location and time of

conflicting/overlapping activities during earthwork operations can also be analysed considering the soil

characteristics along a road section. The system enables project planners and construction managers to take

preventive measures by forecasting the possible conflicting activities in earthwork operations. If planned

productivity varies time progression due to variation in soil characteristics or site access points along the road

projects, there is a chance of overlapping or confliction between activities with different productivity. The

following section describes the detailed derivation of mathematical formula for the identification of conflicts

between two earthwork activities.

FIG. 5 (b): Snap shot of automatic generated time location plan using one set of filling and one set of cutting

equipment for earthwork operations.

3.3 Sensitivity analysis of earthwork duration The sensitivity analysis of earthwork duration presented below demonstrate that the capability of the model to

assist in analysing with “what –if” scenarios for different variable factors including site access points, type of

equipment and soil characteristics along a road section that affect the development of a time location plan and

resource planning in the earthwork operations.

3.3.1 Sensitivity analysis of earthwork duration due to site access points:

The result presented in figure 6 shows that total duration was 7 days for filling-1 & 2 sections and 3 days for

cutting-1 section of earthwork operations, which is similar for access points 3 and 5 assuming that other

variables and resources are constant. However, the total duration was 9 days for filling-1 & 2 sections and 4

days for cutting-1 section of earthwork operations for six number of access points, which are higher than the

smaller number of, access points. Therefore, the conclusion from these results is that lower number of access

point is more economical, less time and resources consumption to complete the same quantity of earthwork in

comparison to a higher number of access points for same section in a road construction project.

FIG. 6: Sensitivity analysis result of total duration for

variable of site access points

FIG. 7: Sensitivity analysis result of total duration

for variable of equipment types

Existing time location plan of cut/fill activity

Filling activity

Cutting activity

204


3.3.2 Sensitivity analysis of earthwork duration due to equipment types

The results presented in figure 7 above show that total duration was 35 days for filling 1 & 2 sections and 18

days for cutting -1 section of earthwork operations at excavator type -1(Exa). Whereas, total duration was 35

days for filling 1 & 2 sections and 14 days for cutting-1 section of earthwork operations at excavator type -2

(Exb) assuming other variables and resources are constant. However, the total duration was 34 days for filling-1

& 2 sections and 12 days for cutting -1 section of earthwork operations when using excavator type -3 (Exc),

and total duration was 26 days for filling -1 & 2 sections and 9 days for cutting-1 section of earthwork

operations using excavator type-4 (Exd). Therefore, the above results revealed that higher productive equipment

is more economical and need less time and resources to complete the same quantity of earthwork in comparison

to lower productive equipment in same section of a road project and under similar circumstances. Since the

mobilisation and demobilisation cost is same for each type of equipment, it will be more economical and logical

to plan and use higher productive equipment for the earthwork planning if site conditions allow the operation of

higher productive equipment.

3.3.3 Sensitivity analysis of earthwork duration due to site soil characteristics

The result presented in figure 8 shows that total duration was 40 days for filling-1 & 2 sections of earthwork

operations since the soil characteristics for all filling sections were same throughout the road section. However,

the total duration for cutting section for different types of soil was different. The result shows that duration of

cutting-1 sections are 20, 19, 21 and 23 for different types of soil for sand, sand-clay, clay dry and clay wet

respectively. Therefore, the above results confirm that sand-clay soil at cutting section requires less time in

comparison with sand, clay-dry and clay-wet whereas soil types such as clay-wet need more time to complete

the same quantity of earthwork under similar site constraints in comparison to other soil characteristics.

FIG. 8: Sensitivity analysis result of total duration for variable of soil types

From above result, it is anticipated that the sensitivity analysis will assist project planners and construction

managers in simulation analysis with “what-if” scenarios for the production of efficient construction scheduling

and resource planning under different site conditions including types of equipment, soil characteristics and site

access points in a road construction project. The results of sensitivity analysis confirm that all three variables

(site access points, types of equipment and soil characteristics) are the most critical variables, which have direct

impact in productivity and resource planning for earthwork operations in a road project. Therefore, project

planners and construction managers need to analysis and simulate more carefully at detailed planning stage for

the production of efficient construction scheduling and resource planning in the earthwork operations in a road

construction project.

4. CONCLUSIONS The paper has introduced an innovative methodology for the development of a prototype model dubbed as

“Virtual Construction Model (VCM)” aiming to produce and visualise earthwork progress profiles and a time

location plan throughout the earthwork operations in road construction projects. The developed model has

capability to generate progress profiles according to “variable” productivity data and at a particular time

considering it 4th dimensions, which is derived from the productivity of earthwork activities. This is considered

205


as an innovative approach for the VCM comparison to 4DCAD technology where variation of earthwork

productivity due to site conditions and soil characteristics was not integrated with the 4DCAD models. The

model also generates visual representation of construction progress and a time location plan of construction

project showing an accurate location of the earthwork activities and corresponding time. The model has

capability in interfacing with user-defined variables including soil characteristics and site access points

according to topographical constraints, which is considered as another key achievement of this research study.

The sensitivity analysis presented in the paper with a case study confirms that the model assists project planners

and construction managers to analyse “what-if scenarios” with soil characteristics and resource constraint

through the visual simulation in construction scheduling and resource planning processes.

The paper concludes that the VCM introduced by the research is a decision support tool for earthwork

construction management. The model will facilitate a logical decision-making process for the earthwork

construction scheduling and resources planning tasks in improving site productivity and reducing the production

cost of earthwork operations in road projects.

5. ACKNOWLEDGEMENTS This research is supported by the Centre for Construction Innovation and Research (CCIR), University of

Teesside, UK and Portugal based International Construction Company, MOTA-ENGIL. Sincere appreciation is

given to the sponsor from the authors.

6. REFERENCES Adjei-Kumi, T., Retik, A. and Shapira, A. (1996), ‘‘Integrating on site tasks into planning and scheduling of

construction projects.’’, Proc., shaping theory and practice, managing the construction project and managing

risk: Int. Symp. on organisation and management of construction, D.A. Langford and A. Retik., eds., E &

FN Spon, Glasgow, U.K., 283-292.

Castro, S. and Dawood, N. (2005), “RoadSim: A Road Construction Knowledge-Based Simulation System”,

Proceedings of the CIB W102 Conference, 2005, Lisbon, Portugal.

Dawood N., Eknarine S., Mallasi, Z. and Hobbs, B. (2002) “Development of an integrated information resource

base for 4D/VR construction processes simulation” Automation in Construction, (12) pp 123-131.

Garold, D.O., Samir, A. A. and Gregory, A. D. (2005), “Linear scheduling of highway project”, Oklahoma

Transportation Centre Research Project, OTC-25.

Harmelink, D. J. and Yamin, R. A. (2000), “Development and application of linear scheduling techniques to

highways construction projects”, Technical report of FHWA/IN/JTRP-2000/21, Oct. 2000, pp8-69.

Johnston, D. W. (1981), “Linear Scheduling Method for Highway Construction”, Journal of Construction

Division, ASCE, 107 (2) pp. 247-261.

Kang, L. S., Moon, H. S., Moon, S. J. and Kim, C. H. (2008), “ 4D System for visualisation scheduling progress

of Horizontal Construction project Including Earth work”, Proceedings of the CONVAR Conference 2008,

21-22 October, Kula Lumpur, Malaysia.

Koo, B. and Fischer, M. (2000), “Feasibility study of 4D CAD in commercial construction”, Journal of

Construction Engineering and Management, 126 (4), 251-260.

Liapi, A. K. (2003), “4D visualisation of highway construction projects” Proceedings of 7th International

Conference on Information Visualisation (IV’03), 1093-9547/03 ©2003 IEEE.

McKinney, K. and Fischer, M. (1998), ‘‘Generating, evaluating, and visualizing construction schedules with

CAD tools’, Automation in Construction, 7(6), 433–447.

Shah, R. K. , Dawood, N. and Castro, S. (2008), “Automatic Generation of Progress Profiles for Earthwork

Operations using 4D Visualisation Model”, Electronic Journal of Information Technology in Construction,

IT Con. Vol. (13), PP 491-506.

Zhang, J. P., Anson, M. and Wang, Q. (2000), ‘‘A new 4D management approach to construction planning and

site space utilization.’’ Proc., 8th Int. Conf. on Computing in Civil and Building Engineering, ASCE,

Reston, Va., 15–22.

206


DEVELOPMENT OF 3D-SIMULATION BASED GENETIC ALGORITHMS TO SOLVE COMBINATORIAL CREW ALLOCATION PROBLEMS

Ammar Al-Bazi, PhD Candidate,

School of Science and Technology, University of Teesside;

[email protected], http://tees.academia.edu/AmmarAlBazi

Nashwan Dawood, Professor,


[email protected], http://tees.academia.edu/NashwanDawood/

John Dean, Project Officer,


[email protected]

ABSTRACT: This paper presents an innovative approach to solve combinatorial crew allocation problems in any

labour-intensive industry. This possibly can be achieved by combining 3D-simulation technology with Genetic

Algorithm (GA). GA is one of the Artificial Intelligent tools that were successfully used in optimising performance of

simulation models. The integrated system can determine the least costly and most productive crews to be assigned to

production processes. Discrete Event Simulation (DES) methodology is used to develop a 3D-simulation model that

conveys the idea of how a labour-driven manufacturing system works. A proposed GA-based Multi Layers

chromosome is developed to be integrated within the developed simulation model. This type of integration can

optimise performance of the developed simulation model, through guiding it toward better solutions. The concept of

Multi-Layers chromosome is proposed in order to store different sets of labour inputs such as (daytime shift crew,

night shift crew, process priority, etc). GA operators are developed to ensure more random search for promising

solutions in a large solution space. As a case study, a sleeper precast concrete manufacturing system is chosen to

prove the concept of the proposed system. The results showed that adopting different allocation plans had a

substantial impact on reducing total allocation cost, process-waiting time, and optimising resource utilisation. In

addition, worker utilisation and process-waiting time have a significant effect on the labour allocation cost.

KEYWORDS: 3D-simulation, Multi-Layers Genetic Algorithms, Crew Allocation Problem, Precast Concrete

Manufacturing System

1. INTRODUCTION

Crew allocation is the process of deciding where and when, crews can be assigned according to their required qualification or skills. Crew consists of a composition of different skilled workers to carry out a certain process in a production facility. This term is used in processes, which needed intensively human resources to carry out jobs.

The complexity of crew allocation problem appears when different production processes demand the same type of resource (worker or machine) at the same time. Such competition on using resources has the potential to cause process-waiting time, labour idle time, low resource utilisation, and subsequently high labour allocation cost. Such allocation problems can be seen in a job-shop environment such as seen in the precast concrete industry.

Many analytical assignment models have been developed to solve allocation problems during the last 50 years, Pentico (2007). Many real life allocation problems are complex and involve a huge number of alternatives to be modelled, this alternative explosion causes a combinatorial problem. To avoid the problem of ‘combinatorial explosion’, many heuristic rules and artificial intelligent tools have been developed to solve such problem.

Crew allocation system seems to be a very visible and economically significant tool to solve such complex allocation problem. The crew allocation system can be used as a decision support system in crew planning and

207


scheduling of labours in many manufacturing systems where human resource is a substantial resource. The use of such an allocation system can assist production managers to identify the best allocation plan in a short time.

In this paper, an innovative crew allocation system dubbed “SIM_Crew” is developed. Process Simulation and Artificial Intelligent technologies have been integrated to produce a sophisticated crew allocation system. GA is suggested as a promising optimisation engine to be embedded within simulation model for better searching. An intelligent searching algorithm is developed to explore a large solution space. Multi-Layers chromosome that can be used to store and manipulate different sets of various attributed data is proposed. A special selection rule is designed to give a higher selection chance for promising chromosomes for further investigation. Dynamic Probabilistic Crossover (DPC) is developed to enable stochastically the exchanging of genes at different chromosome layers. In order to avoid the local minimum trap, avoid chromosome repetition, and to add more randomness to the searching process, a Dynamic Probabilistic Mutation (DPM) is developed to exchange randomly the current proposed crew for a process with any of the crew alternatives for that process. This paper is organised as follows: In section 2, the literature review exploring crew allocation techniques is presented. In section 3, the specification of the crew allocation system is demonstrated. In section 4, a real life case study is presented. Conclusion and future development are the contents of the last section.

2. RELATED WORK ON CREW ALLOCATION PROBLEMS

Crew planning and scheduling using simulation modelling and optimization technology has been charted in the works of Lu, et. al (2005) who presented a computer system called “simplified simulation-based scheduling (S3)” to solve the problem of skilled labourer scheduling in a multi-project context. Marsono (2006) developed a simulation model for the production of Industrialized Building System (IBS) components. Dawood, et. al (2007) developed a generic simulation model depicting the operational processes of precast concrete production systems. The simulation model was developed to study cost and a trade-off schedule under different resource allocation policies, resource utilisation evaluation was considered in this study. Nassar (2005) presented a model that uses spreadsheet GA implementation to optimally assign resources to the repetitive activities of construction projects, in order to minimise the overall project duration as well as the number of interruption days. Ipsilandis (2006) presented a linear programming parametric model formulation for supporting the decisions of construction managers: explored the multi-objective nature of decision-making in repetitive construction projects. Watkins, et al. (2007) used agent based modelling methods to simulate space congestion on a construction site to explore the impacts of individual interactions on productivity and labour flow. In this simulation, two masonry crews intersecting in space are considered. Li, et. al (1998) presented a methodology for optimising labour and equipment assignment for excavation and earthwork tasks using a Genetic Algorithm. Moselhi et. al (2007) proposed a new methodology which uses combined Genetic Algorithms and spatial technologies for optimisation of crew formations for earthmoving operations. Marzouk, et. al (2007) presented a special purpose simulation model to capture the uncertainty associated with bridge construction. A sensitivity analysis was performed to study the impact of assigned labour crews in the estimated durations of segment fabrication and deck construction. The studies above presented a number of methodologies and tools that have been developed so far to manage and schedule crews in a number of labour-intensive industries.

More systems that are sophisticated seem to be needed to advance the practice of crew planning and scheduling in any labour-intensive industry. As more analyses are required to identify the behaviour of allocation solutions, analysis of effects of resource utilisation and other performance indicators on reducing labour allocation cost are required to be enabled in such sophisticated systems. In the next section, the specification of the proposed allocation system and “SIM_Crew” modules will be described in detail.

3. THE CONCEPTUAL MODEL OF THE PROPOSED ALLOCATION SYSTEM

Conceptual model can be defined as a visual diagram of representing a set of causal relationships among proposed components that are believed to form a desired integrated system for solving a particular problem.

The purpose of developing a conceptual model is to capture all possible causal relationships among the real-world entities, which are concerned with the core functionality of the system being proposed. The process of constructing a

208


conceptual model starts with an analysis of the entity requirements of the real world, to identify those entities which are of key significance to the system, excluding any that are irrelevant or unrelated to its core functionality.

In this study, a conceptual model is proposed to introduce an innovative crew allocation system, this model shows how all components of the integrated system can work together in order to process inputs and produce outputs. The purpose of manipulating crew allocation is considered as a trial to place the right and most suitable crew to a process so minimum allocation cost is achieved. Therefore, as a fundamental requirement, crews should be allocated to production processes with minimum delays that might caused by shared labourers of different crews to carry out same job at the same time. In addition, the performance of alternative feasible allocation plans should be evaluated in terms of labour allocation-related cost, so that the efficient one with the minimum labour allocation cost and reduced process-waiting time is selected, for implementation as the best crew allocation plan. To satisfy the above allocation process objective, system specification is developed which involves integration of simulation technology and Genetic Algorithms as a processing core shown in figure 1.

FIG. 1: “SIM_Crew” Conceptual Model

In figure 1, “SIM_Crew” consists of a simulation model, which is integrated with databases through the integration and processing module (this module is developed by writing a VBA codes). Two types of databases are developed to provide the simulation model with the required information about product specifications and other related labour information. An optimization module is designed to be embedded in the simulation model: the function of this module is to provide simulation with high quality feasible allocation plans for evaluation purpose.

Many key performance indicators such as labour allocation costs, process-waiting time, and utilisation of labourers are considered as outputs. The interface mechanism is designed to include size of population, selection strategy, optimisation engine operators (crossover and mutation), and other requirements. As shown in figure 1, the allocation process is an iterative procedure of progressive improvement in which GA module proposes more than one allocation plan to be evaluated by the simulation model, results are fed back to the optimisation engine to decide and propose according to the efficiency more promising allocation plans. During allocation iterations, simulation executes allocation plans: each of them consists of a set of proposed crews to be allocated to production processes, while GA evaluates the performance of the resultant allocation, and based on this, adjusts the decision variables and selects the most promising. A conceptual flexible model for development of intelligent simulation in manufacturing systems was presented by Azadeh, et. al (2006). The SIM_Crew modules will be described in the following sub-sections:

3.1 Constructing a Relational Database Model

In order to ensure a quick locate or access to any specific crew’s member, crew alternatives and other worker specifications, a relational database model is developed for such purpose. In addition, the purpose of developing a relational database model is to structure the storage of information in a way that future development such as adding

209


or removing a process, worker, or any other related detail would be easier and flexible. In addition, Structured Query Language (SQL) facility, which Access database can provide was useful to ease the searching for any labour information. SQL in the optimisation phase is useful to arrange in an ascending order the resulting costs so that selection of minimum cost chromosomes over all generations is enabled. The developed relational database enables the designed system to be more flexible in terms of adapting any number of processes, any number of shifts, any number of crews, and any number of workers. By having lists (workers’, crews’, or processes’ lists) that cross-reference each other, a flexible scenario can be built and created.

3.2 Modelling Phases of a 3D Simulation Model

The methodology used to develop a 3D simulation model starts with identifying the logic of labour-driven processes. Many structured interviews with production managers and senior workers beside a number of on-site visits are conducted, to capture the hierarchy of such production system and to identify the interchanging relationships among its components. The logic of production processes was identified and a flowchart was prepared by Al-Bazi, et. al (2009). Figure 2 shows the development phases of the 3D simulation model.

FIG. 2: development phases of a 3D simulation model

After identifying the logic of the processes, the simulation modelling process starts by translating the static version of the process logic into a dynamic simulation model. ARENA SIMAN language is used to enable such translation by adopting Discrete Event Simulation (DES) concept. The resulting model consists of simulation blocks that linked with each other by using smart links. Those blocks involve decision, process, assign and other useful modelling blocks that ARENA software provided. The visualisation of simulation model starts by using the available modules from the 2D library. Such animation gives a 2D representation useful to identify the verification of the model. The 3D visualisation is developed using advanced 3D modules which 3D-ARENA provided. The purpose of such modelling is to provide a navigation tool that enables users or production managers to explore the virtual model’s components and to see whether it has been successfully represent the real world manufacturing system or not.

3.3 Developing an Intelligent Searching Algorithm: Genetic Algorithms

Genetic Algorithm (GA) works with a population of solutions rather than a single solution. In this problem, an optimal allocation plan needs to be chosen from a large pool of plans. Genetic Algorithms deal with this sort of searching rule as the population of crews can be coded in terms of chromosomes, and then each chromosome can call the desired crew by referring to its index. The main innovative contribution of a GA is the novel construction of a neighbourhood based on natural selection principles, Ólafsson, et. al (2002).

210


3.3.1 Multi-Layers Chromosome Engineering

For a Multi-Layers chromosome, all genes have given sequential numbers, started from 1 to the number of the last gene allocated in the last layer of the chromosome. This sort of Multi-Layers chromosome structure is proposed and developed in order to have less parameter to pass through while coding the program and to make comparison easy. See figure 3

FIG. 3: mutli-Layers chromosome structure

The idea behind designing a Multi-Layers chromosome is to present how different attribute labour inputs can be stored in a chromosomal structure design. This structure is useful to present different sets of inputs in terms of multiple layers so it can coded easily and presented better. The nature of inputs for the “SIM_Crew” model drove the developed chromosome to adopt Multi-Layers structure rather than a conventional form. Each set of inputs involved production process ID, number of crews available to carry out each process and the shift type. For each process, possible alternative crews involving all required skilled of workers are stored into a pool of crew alternatives. The first layer of the chromosome involves all possible daytime crews. The second chromosome layer is assigned to accommodate nightshift crews. The designed chromosome enables Structured Query Language (SQL) to guide the searching of any process to find a feasible allocation plan. Each process may involve more than one crew in which any of them is able to carry out the process with different crew formation and within a different processing time. Each process can be carried out using more than one working shift to satisfy commitments with clients, so not all processes are necessarily to have the same number of working shift processes. The developed Multi-Layers chromosome enables each set of data relating to a working shift to be placed into a specified Layer.

3.3.2 Population Initialization

The initial population, which is then evolved by GA, should, if possible, be well spread through the search space, so that it is well sampled. Random sampling is used to generate the initial population.

As one of the random sampling techniques, Monte Carlo (MC) simulation is used to generate crews’ indices. For each gene, an integer random number represents position of the crew’s alternative is generated to select randomly the crew’s alternative of each process.

3.3.3 Objective Function of SIM_Crew

The purpose of allocating crews is to optimise the manufacturing system performance; thus, manufacturing system performance such as manufacturing time, production cost obtained from a running simulation is used as the fitness

211


value within the GA operation. The objective function is applied to evaluate the total resource costs. The equation used to calculate such objective function is:

(1)

Where:

: cost objective function values obtained by evaluating chromosome i

n : the number of labour-driven processes

iBRC : incurred cost per hour when using a labourer for set of solution i.

iIRC : incurred cost per hour when labourer is idle for set of solution i

iRCPU : incurred cost per use of fixed or physical resource for set of solution i,

senior skilled bonus can be considered in such cost.

In some situation, there is a trade-off exists between idle time and cost (El-Rayes, et. Al 2001). In this study, only direct cost is considered as a substantial cost.

Calculation of Fitness Function

For minimisation type problems, modifying objective function is required to give high weights for minimum costs to be selected for further evolution. The modified objective function is called fitness function, the fitness function expression is:

(2)

Where:

Fitness function value for chromosome i

Largest cost in population i

3.3.4 Development of the Genetic Algorithms Operators

For the allocation system being developed, GA operators have developed to solve this type of allocation problem. Selection, crossover and mutation strategies are developed to achieve a better search for promising solutions. Al-Bazi, et. al (2009).

The Proposed Class Interval Selection Rule (CISR)

In this proposed rule, only the promising chromosomes with least costs or higher fitness functions will be considered as a potential improvement vehicle. The proposed selection rule is named “class interval” rule, which is developed to provide the promising chromosomes with higher probability of selection to produce good solutions. The main concept of this selection rule depends on constructing a “class interval” which is used in descriptive statistics. Repetition of any generated chromosome is not allowed, so all generated chromosomes should be unique all over the evolution process,

Probabilistic Dynamic Crossover (PDC) Strategy

The crossover operation in a conventional GA is based on the exchange of genes between two fixed length chromosomes when coding is applied for chromosomes. To crossover genes in the chromosome, (0-1) variates should be generated for each gene in the Multi-Layers chromosome. This type of exploration investigate all active genes (occupied genes by scheduled crew with a shift) for more randomness. A random number is then generated to exchange genes after satisfying a certain condition. In this strategy, random numbers are generated to be associated with each gene at each layer, if the gene is vacant for a reason (no applied shift if there is no crew) then the generated random number will be discarded to skip to the next gene.

212


Probabilistic Dynamic Mutation (PDM) Strategy

To avoid local maxima and to randomise the searching process, a modified mutation process is developed to swap the gene within a chromosome with its available set of alternatives. This strategy is similar to PDC but the only difference is that Monte Carlo (MC) sampling applies to search stochastically for a crew within each assigned pool of crews. Each offspring (an individual chromosome) is randomly selected and genes are mutated vertically with its set of alternatives using Monte-Carlo sampling.

4. CASE STUDY

A real life case study for one of the largest sleeper precast concrete manufacturer in the UK was developed. The purpose of developing this case study is to identify the performance of the proposed allocation system. In sleeper manufacturing system, the production process involves using of a wide range of different resources, including labour, equipment and materials. Each production section has two labour-intensive production lines; shared resources (workers and machines) are used to carry out job/activity on each production line alternately. Eight processes including curing process are applied on the sleeper product being produced to deliver the final product. Similar labour-intensive processes are adopted at each production line, processing time depending on the type of product being produced at any production line.

A simulation model is developed using ARENA simulation software to simulate the production processes of the sleeper concrete manufacturing system. All relevant data was collected by conducting onsite visits, using flowcharts, interviews and stopwatch technique as relevant data collection techniques. More detail about crew formation data, worker details are confidential and has not provided in this study. See figure 4 for the 3D animation of the sleeper manufacturing system considered in this study.

FIG. 4: snapshot of the sleeper precast manufacturing system

In figure 4, all shared resources such as casting machines, run strand wire car and stress machine are animated to show how such productive resources can be shared to carry out a process visually.

4.1 Experimental Analysis and Evaluation

The experimentation part of this study involves allocating possible crew alternatives to production process in order to identify minimum allocation cost. In this case study, Multi-Layers chromosome structure is developed to store crew alternatives, processes and shift patterns. The parameters used in GA are carefully selected by design of experiments. It has been noticed that manipulation of parameters makes no significant differences in terms of solution quality. The best parameter settings are thus identified as follow: the population size is found to be 20. The gene crossover probability is identified to be 0.70 and gene mutation probability is 0.90. Number of processes is 28 processes and 66 resources (workers and machines) are utilised in this job-shop. Two working shifts are applied in section 1 and one shift is in section 2. The stopping condition is satisfied when there is no reduction in the resulting cost for five consecutive generations (100 chromosomes). A number of scenarios are developed to evaluate

213


performance of various allocation plans. The resource allocation cost is selected as a criterion of such scenarios. The first scenario is “Before Cost Drop” in which the allocation plan before the first cost drop is identified. The allocation plan obtained after the cost drop is called “After First Cost Drop” scenario. The current adopted scenario is called “As-Is”. The scenario with the minimum allocation cost is called “Best Allocation Plan” followed by “Worst Allocation Plan” obtained by adopting maximum allocation cost. Figure 5 shows the reduction in cost using SIM_Crew allocation system.

FIG. 5 cost reduction using SIM_Crew system

It has been noted that the results tend to be stable after 50 generations; a significant reduction in crew allocation cost is obtained. This significant reduction occurred early at the beginning of the generations and after generation 20. This reduction took place because GA operators have successfully explored more solutions that are promising and provided the required randomness in the mentioned generations. The PDM operator played a vital role in bouncing the solution out of local minimum traps. The high probability of gene mutation kept crews manipulation of most processes (genes) active, which eventually can identify more promising solution.

4.2 A Comparison Study between All Scenarios: Process-Waiting Times

A comparison study is conducted to identify the effect of adopting different allocation plans on the crew allocation cost. Average process-waiting time is selected as a substantial factor to compare these allocation plans in order to come up with the best plan. Figures 6 and 7 show comparison of average process-time obtained after running each scenario.

FIG. 6: Comparison of average process-waiting

time achieved in production section 1

FIG. 7: Comparison of average process-waiting

time achieved in production section 2

214


Figures above reveal that the “best allocation plan” scenario achieved minimum average process-waiting time. The highest process-waiting time is achieved by running the “worst allocation plan” in section 2. In addition, “before

first cost drop” has a process-waiting time close to the “worst allocation plan” scenario. The minimum process-waiting time obtained in both sections led to a minimum crew allocation cost.

The “As-Is” scenario considered as the second best scenario, the process-waiting time obtained using this scenario was the second best waiting time. It has been concluded that idle time has a significant impact on the total crew allocation cost as idle time is considered as a substantial factor in the total allocation cost function.

5. CONCLUSION AND FUTURE DEVELOPMENT

The proposed allocation system “SIM_Crew” showed that it was possible to minimise crew allocation cost by adopting an innovative allocation system. The developed simulation-based GA can provide an optimised crew allocation plan to any labour-intensive industry. Probabilistic dynamic crossover and mutation operators were developed successfully to explore the solution space by avoiding local minima solutions. After running SIM_Crew, the results showed a significant cost reduction by allocating the proper crew to the right process and take into account minimising process-waiting time. The developed operators played a vital role in adding more randomness to the searching process. The new concept of using a GA in the crew allocation process and the developed Multi-Layers chromosome application has been proven as a sophisticated and advanced technique through this case study. Different levels of priority for each production process can be included as a future work when designing the chromosome. Multi-objective optimisation is also important in order to model this type of allocation problem.

6. REFERENCES

Al-Bazi, A. and Dawood, N. (2009).“Decision support system for pre-cast concrete manufacturing planning: An innovative crew allocation optimiser”. The 2009 CSCE International Conference on Computing in Civil

Engineering, St. John's, Newfoundland & Labrador, Canada.

Azadeh, A., and Ghaderi, F. (2006). “A Framework for Design of Intelligent Simulation Environment”. Journal of

Computer Science 2 (4): 363-369, 2006

Dawood, N, Ahmed, R. and Dean, J. (2007). “Modeling of Precast Concrete Production Operations and Innovations: A Simulation Approach”. Manubuild Conference, Rotterdam, 25-26 April.

El-Rayes , K., and Moselhi, O. (2001). Optimizing resource utilization for repetitive construction projects. Journal

of Construction Engineering and Management, Vol. 127, No.1, pp. 18-27

Ipsilandis, P.G. (2006). “Multi-objective optimization in linear repetitive project scheduling”. Operational Research.

An International Journal. 6(3): 255-269.

Li, H., Love, P., and Ogunlana, S. (1998). “Genetic algorithm compared to nonlinear optimization for labor and equipment assignment”. Building Research and Information, 26(6): 322-329.

Lu, M. , and Lam, H. (2005). “Optimised concrete delivery scheduling using combined simulation and genetic algorithms”. In Proceedings of the 2005 Winter Simulation Conference, M. E. Kuhl, N. M. Steiger, F. B.

Armstrong, and J. A. Joines, (Eds.) 2572-2580. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers, Inc.

Ólafsson, S. and Kim J. (2002). “Simulation Optimisation”. Proceedings of the 2002 Winter Simulation conference

Pentico, D.W., 2007, “Assignment problems: A golden anniversary survey,” European Journal of Operational

Research, Vol.176, No.2, pp.774-793

Watkins, M., Mukherjee, A., Onder, N., and Mattila, K. G. (2007). “Understanding Labour Productivity as an Emergent Property of Individual and Crew Interactions on a Construction Site”. In Processing’s of the IGLC-

15, July 2007, Michigan, USA: 400-405.

215


Marsono, A. , Masine Md. Tap, Ching N. S. and Makhtar, A. M. (2006). “Simulation of Industrialized Building System Components Production”. In Proceedings of the sixth Asia-Pacific Structural Engineering and

Construction Conference (APSEC 2006), Kuala Lumpur, Malaysia

Marzouk, M. Hisham Zein El-Dein, and Moheeb El-Said. (2007). “Application of Computer Simulation to Construction of Incremental Launching Bridges”. Journal of Civil Engineering and Management, Volume XIII, Number 1, pp. 27-36.

Moselhi, O. and Alshibani, A. (2007). “Crew optimization in planning and control of earthmoving operations using spatial technologies”. IT Construction, 12: 121-137.

Nassar, K. (2005). “Evolutionary optimization of resource allocation in repetitive construction schedules”, IT

Construction, 10: 265-273.

216


INTEGRATION OF URBAN DEVELOPMENT AND 5D PLANNING

Nashwan. Dawood, Professor

Centre for Construction Innovation & Research, Teesside University, UK

[email protected]

Claudıo. Benghi, Dr

Built Environment, Northumbria University

Thea. Lorentzen, and Yoann. Pencreach

FORUM8, Japan

ABSTRACT: 5D planning is the process of integrating 3D models of buildings or infrastructure projects with

construction activities and cost planning. The process has been used successfully to rehearse the construction

process and indentifying ‘hot spots’ and process clashes prior to site construction activities. Previous research

(Dawood, 2008) has concluded that at least 7% of construction time and cost can be reduced if the technology has

been deployed at an early stage of construction process. However, construction planning within the context of urban

development not fully exploited and tools and methods for synchronising and rehearing multiple construction

planning within urban setting and identify, for example, traffic congestions and other environmental issues that can

be affected with construction processes and nearby sites. In this context, the aim of this paper is to deploy and

develop 4D process and technology for urban planning and construction. The objective is to rehearse construction

processes for urban construction which can involve a multiple of construction projects, both buildings and

infrastructure can be constructed concurrently and can cause massive disruption and congestion at urban scale.

Traffic management and flow can be incorporated within the urban simulation and therefore congestions caused by

multiple construction sites can be identified and resolved before construction starts.

This paper presents a framework and tools for rehearsing multiple construction projects that was developed to

identify issues and hot spots at urban scale. The paper also present initial results of integrating Uc-win/Road visual

urban planning with nDCCIR 5D planning tool. A simple case study was used to demonstrate the technology.

KEYWORDS: 4D planning, Urban Planning, Virtual Reality

1. Introduction to nD planning and urban developments

Recent work in urban planning, developments and transportation visualisation is evolving from a focus on how

projects look to a desire to see how they actually work, i.e. the interaction between different objects and the

influence of different risk factors on urban developments. As many walk-through simulation methods focus on a

scene’s aesthetic qualities, there has been a growing need for visualization of processes. As such, developments in

virtual reality simulation come as a result of an increasing recognition of the value of visualization for representing

not just infrastructure, but “operations and in particular construction”.

Virtual reality is a technology that enables users to interact with a simulated world. When applied to transportation,

VR has the potential to not only model traffic flows; it also allows for construction simulation in the form of nD

modelling. Observing built and natural environments from the construction team provides a viewpoint that is often

ignored in traditional planning methods. Particularly for cases where construction can affect traffic flow in an urban

scene, health and safety issues, portfolio investment and others.

The purpose of this system development is to implement 5D planning so that it can serve as a common visual and

experiential language for presenting, discussing, and ideally improving civil engineering and building construction

projects. The UC-win/Road software was first launched in 2001 and initial versions were primarily used as tools for

visualizing alternative road designs. With an engineering platform, the program continues to be used as a visual tool

for engineers to discuss traffic flows, road alignments and land-use issues. Recent versions have been adapted to

include 5D models of construction projects, animated traffic and human characters. The following section introduces

the concept of 5D models and urban development and the integration of these tools is explained and discussed.

217


1.1 5D project control

Multi-dimensional project control, also known as 5D Project Control, consists of enriching 3D model-based

construction information with data from other domains. In current literature, the 4th dimension is always referring to

time/schedule information while the 5th is commonly considered to be cost but may be related to other domains

such as risk, task ownership, products durability, etc.

4D construction process ideas have been developing for more than ten years by many academics and research

groups (McKinney, K. et al., 1996). 4D modelling is often related to advanced visualization techniques; the first

achievement of 4D modelling was the 3D visual animation of construction sequences through time allowing an

effective representation of inherently abstract scheduling data which held by Gantt charts in a inexpressive form.

Although not standardized or widespread on the market, 4D modelling traditionally refers to the ability to perform

such a visual rehearsal of construction planning, often according to the workflow presented in figure 1.

FIG. 1: Process flow for traditional 4D rehearsal

In the represented process “Planning” and “3D drawing” are connected because the CAD model often needs to be

organized accordingly to the determined planning sequences before the actual linking process can begin; when this

is the case 3D objects are usually grouped in CAD layers accordingly to the planned activities and finally activity

names or codes are mapped to the corresponding layers. This process requires ad-hoc and time consuming drafting

and linking sequences.

The availability of integrated engineering design tools, including Building Information Modelling applications,

provides a unique authoritative repository for the storage of all the design information needed for the definition of

the construction programme. Exploiting such information, a paradigm shift is envisioned to use extended 5D tools

not just for programme but for integrated programme development (Tse et al. 2005)

FIG. 2: Process flow of theoretical integrated construction management

218


The present paper presents the rationale and half-term results of a two year programme aimed at developing methods

and custom IT tools to exploit the potential of such data repositories in the development of cost-aware schedule

programmes in the Oil and Gas industry.

1.2 Urban planning using VR

Virtual modelling of urban planning can provide architects, planners, investors, and managers a collaborative,

future-oriented tool. Current digital urban modelling projects such as Virtual Los Angeles and Virtual Kyoto are

especially focused on bringing together resources and processes to provide aid in disaster management planning. For

example, the Los Angeles model aims to provide information not only to urban planners but also to emergency

response teams and government entities (Virtual Los Angeles). The ability to model several processes at the same

time allows for scenarios such as building evacuations or traffic accidents to be studied prior to their occurrence

(i.e..real-time traffic input for evacuation during hurricanes.)

In this sense, virtual project modelling can provide a platform for users to be not only informed, but also engaged in

project plans. Ideally, project time can be reduced if stakeholders are involved and less resistant throughout the

approval process. Also, with a variety of participants simultaneously negotiating a plan, earlier identification of

problems and objections as well as better informed decisions can assist in reaching project goals (Walker, 2002).

When applied to urban spaces, virtual reality has the potential to create a platform that allows users to not just view,

but interact with a highly accurate visualization of transportation agents and built environments. Proposals can be

juxtaposed against existing structures. The current VR system has been used to create a digital model of Phoenix,

Arizona with the goal of goal of mitigating urban sprawl problems by facilitating smarter planning. Similarly, the

program is being used to communicate with citizens about the construction of a new LRT route and urban renewal

program in Sakai City, Japan. The VR environment is able to reproduce different time intervals and scenarios,

allowing presentations and interactions to encompass changes in space, time and activity. The potential development

of these tools with multi-dimensional project control is fully explained in the next section.

2. RESEARCH THEMES

The use of multidimensional project control systems (5D Planning) has proven benefits in a number of different

indicators of the procurement process value: On site productivity, Team integration, Client satisfaction, Planning

efficiency, Number of rework, Communication, H&S related risk and Construction time; the advantages of all these

sum up to an estimated 7% in terms of expenditure savings (Dawood and Sikka, 2006).

Current multidimensional research roadmap at CCIR (see Figure3) focuses on the development and evaluation of

integrated tools across all of the procurement process; the vision is to develop a modular and integrated platform

taking advantage of Rapid Application Development frameworks (RAD), available open source software

components and open formats for interoperability and data exchange to extend the role of available tools in the

direction of a tighter integration of design, planning, construction and urban planning. The next paragraphs will

describe the objective of the active research topics.

Procurement

Stage

Functions of current

tools

Active research topics Further development

Design ‐ Semi automated cost and plan generation

Planning ‐ Schedule

rehearsal

‐ Cost integration

‐ Scenario evaluation

‐ Integrated plan

development

‐ Intelligent programme optimization

‐ Space resource optimization

‐ Integration with urban planning

Construction ‐ Schedule

communication

‐ Programme

control and

adjustment

‐ Sub‐contractors schedule

synchronization

‐ Identify traffic congestions

Operations ‐ H&S Control and training

FIG. 3: Medium term multidimensional research roadmap at CCIR

219


2.1 Previous development

The development of the new platform is based on the experience gathered in almost a decade of research in the field

of multidimensional construction management including the development of VIRCON (Dawood and Scott, 2005)

and 4DCCIR (CCIR, 2007). Although available tools already allowed performing plan rehearsal and communication

a new framework has been designed to incorporate the experience gathered during the projects and to allow the

implementation of the functionalities mentioned in the next paragraphs and improve scalability for further

extensions.

2.2 Cost and integration

The availability of Cost information in the 5D model is a major step towards the complete integration of most

relevant project data. For the purpose of multidimensional project control, two types of cost are identified:

• Costs of resources required for the construction, including labour and material resources; and

• Costs associated with building components and materials.

The former are usually managed in Project Management (PM) application in association with project resources; the

latter could be obtained merging quantity information either available in the engineering information or extracted

from 3D models with unit costs available in material and cost libraries.

FIG. 4: Components and processes cost management

To integrate PM cost management, project management data importers had to be extended to read resources and

their usage information from Primavera P3 and P3e and Microsoft Project 2003 and 2007. Cost information from

these applications can be referred to model components through the links that are established with the activities for

4D rehearsal, so that no additional task is required from users to link them.

Development of the cost libraries integration is currently under development along with the interfaces required to

specify engineering information required to correctly specify quantities and related unit costs. The planned

development though includes the ability to perform an automatic identification of the associated costs in the library

where appropriate engineering information is available within the 3D model (i.e. for BIM models).

Integrated cost information allows project managers to

• visually assess cost and cash-flow information in association with the project stages;

• improve assessment of the financial relevance of building components and processes through colour-

coding;

• improve communication with clients; and

• Allow budget control and management.

220


3. SCENARIO EVALUATION

One of the limits identified by previous research was the need to continuously switch between different applications

to improve the schedule as rehearsal helped identifying errors in the programmes. None of the reviewed

multidimensional tools offers the ability to modify schedule data can only import it from planning applications.

Adding the ability to modify programmes interactively is not a trivial task because programme activities have

repercussion on each other when activities dependencies are defined. A scheduling solution engine had to be

developed to evaluate the changes deriving from changing activities dates or durations and their dependencies.

Compared to well establish commercial applications, the developed schedule solution engine is not complete;

however it allows us to perform integrated what-if scenario evaluation within the interface thus saving much time

and immediately seeing the consequences on each programme variation. The application keeps track of all the

modifications the user performs on the tasks and relations so that the same changes can then be applied in the

original project management environment for a data sanity check.

3.1 Urban planning and integration

The micro simulation .xml file format allows for position and movement information for various urban elements to

be displayed together in the same space. Several simulations can be opened at the same time and a playlist manages

their relative positions in time. For example, construction processes, traffic flows, pedestrian movement and

environmental / lighting changes can be shown in the same VR model. The behaviour of these elements can also be

recorded and played back, also allowing for fast forwarding, pausing and jumping though time. Logs of position

information, etc. can be exported for later analysis. When several simulations are shown simultaneously, the

interplay of elements within one urban “world” can be tested so as to avoid collisions in real life. As the micro

simulation format is an open one, we expect that behaviour of other location-specific elements, such as

environmental disasters or material cost information will be added by different users. Main features of the micro

simulation include:

Playlist

Manages the relative position in time amongst several simulations.

Sets time unit conversion for each simulation (e.g. 1 day played in 1 second)

Playback

Support of various time units (nsec to year). By default the original time unit is converted to second

for playback.

Standard playback support: Play, Stop, Fast forward, Pause, Jump…

3.2 Current development

Ongoing development also includes the ability to keep track of updated progress/activity information and to

compare it with the saved baseline to achieve a comprehensive progress control system allowing the Project

Manager to:

• visually compare the expected status of the construction with the current on-site situation at any given

moment in order to correctly assess activity completion percentages;

• rehearse the execution of the tasks due in the following weeks in order to identify ahead previously

undetected conflicts;

• promptly react to any programme change being able to improve assessment of the consequences

deriving from delayed activities, on site problems or programme variations; and

• Effectively manage task reallocation depending on budget constraints and policies.

• Integration with visual urban planning to identify traffic congestion and rehearse of construction plans

of multiple projects.

221


4. IMPLEMENTATION

The discussed project has been developed for the Windows platform on the Microsoft .Net Framework as a multi-

tier application according to the scheme reported in figure 5.

FIG. 5: Development framework components grouped by tier

222


FIG. 6: A screenshot of the application in project control mode within urban setting with

223


5. CONCLUSIONS

Although formal evaluation of the developed tool is still to be undertaken, significant interim conclusions can be

drawn:

• data import functionalities from PM applications and Open 3D formats have been tested extensively

and have proven to provide all the required information;

• public domain software development kits for the gaming industry along with the diffused availability

of accelerated 3D graphic adapters provide suitable frameworks for the development of 3D and

possibly Virtual Reality visual environments;

• The development of the schedule solution engine allowed us to include methods for the rapid

evaluation of alternative programmes within the rehearsal interface and could ultimately result in the

development of an integrated multidimensional programme development environment.

• the use of publicly available Open Source components in Rapid Application Development

environments allows effective application development of articulated frameworks;

6. REFERENCES

4DCCIR, CCIR, (2007). Available at: http://www.idproplan.co.uk/4DCCIR.html [Accessed July 10, 2008].

AGA Treeviewadv, (2007), Available at: http://sourceforge.net/projects/treeviewadv/ [Accessed July 10, 2008].

Al-Kodmany, K. (1999). “Using visualization techniques for enhancing public participation in planning and design:

process, implementation, and evaluation”, Landscape and Urban Planning, 45, pp. 37–45.

Bentley, (2007), “Engineering Project Schedule Simulation”. Available at: http://www.bentley.com/en-

US/Products/ProjectWise+Navigator/Schedule+Simulation.htm [Accessed July 11, 2008].

Bentley, (2008), “Bentley Acquires Common Point to Mainstream Construction Simulation”. Available at:

http://www.bentley.com/en-US/Corporate/News/Quarter+2/Common+Point.htm [Accessed July 11, 2008].

Cadalyst, (2007), “Autodesk Acquires NavisWorks”. Available at:

http://aec.cadalyst.com/aec/article/articleDetail.jsp?id=430659 [Accessed July 10, 2008].

Dawood, N. and Sikka, S., (2006). “The Value of Visual 4D Planning in the UK Construction Industry”. In

Intelligent Computing in Engineering and Architecture. pp. 127-135.

Dawood N, Scott D., (2005). “The virtual construction site (VIRCON) tools: An industrial evaluation”. Available at:

http://www.itcon.org/cgi-bin/works/Show?2005_5 [Accessed July 10, 2008].

Dockpanelsuite, (2007). “The docking library for .Net Windows Forms”, Available at:

http://sourceforge.net/projects/dockpanelsuite/ [Accessed July 10, 2008].

Laiserin, J., (2008). “Vico Virtual Construction Suite 2008” (Cadalyst Labs Review) - CAD Management. Available

at: http://management.cadalyst.com/cadman/article/articleDetail.jsp?id=526884 [Accessed September 11,

2008].

LZMA SDK (Software Development Kit), (2007), Available at: http://www.7-zip.org/sdk.html [Accessed July 10,

2008].

McKinney, K. et al., (1996). Interactive 4D-CAD. In Computing in Civil Engineering. Atlanta, GA: Georgia

Institute of Technology, pp. 383-389 .

Osborne A, (2007). “Commonpoint data integration tools and methods”. Personal interview with Commonpoint Inc.

Vice President – Sales and Marketing

Stowe, K., (2008). “Autodesk - Revit Extension”. Telephone interview with Autodesk In. Construction Business

Development Manager

224


Tse T K, W.K.A.A.W.K.F., (2005). “The utilisation of building information models in nD modelling: A study of

data interfacing and adoption barriers”. Available at: http://www.itcon.org/cgi-bin/works/Show?2005_8

[Accessed July11, 2008].

Walker, D., (2002). “Visualization as a Common Language for Planning: Good Practices, Caveats, and Areas for

Research”, TR News, Going Public: Involving Communities in Transportation Decisions, Transportation

Research Board of the National Academies, 220, pp. 7-11, 2002.

Virtual Los Angeles, University of California, Los Angeles

http://www.ust.ucla.edu/ustweb/projects.html

Virtual Kyoto

http://www.geo.lt.ritsumei.ac.jp/uv4w/frame_e.jsp

http://www.ritsumei.ac.jp/eng/newsletter/winter2006/gis.shtml

Center for Advanced Transportation Technology, University of Maryland

http://www.catt.umd.edu/research/index.html

www.openmicrosim.org

225

SIMULATION AND ANALYSIS

A Simulation System for Building Fire Development and

the Structural Response due to Fire----------------------------------------------------------229

Zhen Xu, Fangqin Tang and Aizhu Ren

Physics-based Crane Model for the Simulation of Cooperative Erections-----------237

Wei Han Hung and Shih Chung Kang

Interaction between Spatial and Structural Building Design:

A Finite Element Based Program for the Analysis of

Kinematically Indeterminable Structural Topologies-------------------------------------247

Herm Hofmeyer and Peter Russell

Virtual Environment on the Apple iPhone/iPod Touch-----------------------------------257

Jason Breland and Mohd Fairuz Shiratuddin

3D Visibility Analysis in Virtual Worlds: The Case of Supervisor---------------------267

Arthur van Bilsen and Ronald Poelman

Evaluation of Invisible Height for Landscape Preservation

Using Augmented Reality-----------------------------------------------------------------------279

Nobuyoshi Yabuki, Kyoko Miyashita and Tomohiro Fukuda

An Experiment on Drivers’ Adaptability to Other-hand Traffic

Using a Driving Simulator----------------------------------------------------------------------287

Koji Makanae and Maki Ujiie

C2B: Augmented Reality on the Construction Site----------------------------------------295

Léon van Berlo, Kristian Helmholt and Wytze Hoekstra

Development of a Road Traffic Noise Estimation System

Using Virtual Reality Technology-------------------------------------------------------------305

Shinji Tajika, Kazuo Kashiyama and Masayuki Shimura

Application of VR Technique to Pre- and Post-Processing for

Wind Flow Simulation in Urban Area--------------------------------------------------------315

Kazuo Kashiyama, Tomosato Takada, Tasuku Yamazaki, Akira Kageyama,

Nobuaki Ohno and Hideo Miyachi

Construction Process Simulation Based on Significant Day-to-day Data-------------323

Hans-Joachim Bargstädt and Karin Ailland

Effectiveness of Simulation-based Operator Training------------------------------------333

John Hildreth and Michael Stec


A SIMULATION SYSTEM FOR BUILDING FIRE DEVELOPMENT AND THE STRUCTURAL RESPONSE DUE TO FIRE

Zhen Xu, Doctoral candidate,

Department of Civil Engineering, Tsinghua University, Beijing, P.R. China;

[email protected]

Fangqin Tang, Postdoctoral,


[email protected]

Aizhu Ren, Professor,


[email protected]

ABSTRACT: The software on simulation of building fire and the software on simulation of structural response due

to fire both have good performance in their respective fields, but they are not good at the synchronous simulation

between building fire and structural response due to fire. In this paper, the synchronous simulation above is made in

the virtual reality system based on Vega. This system is a post-processing platform based a fire simulation software

FDS and a FEM software MSC.MARC. FDS and MARC provide scientific results for synchronous simulation, while

this virtual reality system shows these results in a visual and lifelike way. This system reveals the link between

building fire and structural fire response. In virtual training, this system can help people judge the structural safety

by the situation of building fire.

KEYWORDS: Building fire, Structural fire response, Synchronous simulation

1. INTRODUCTION

Fire is one of the most dangerous disasters to destroy the buildings. The fire safety of building structures is therefore

paid more and more attention to. With the application of the virtual reality technology in more and more fields,

many scientists have studied on the simulation of building fire, such as fire numerical model (Fu Zhuman and Fan

Weicheng, 1996; Yao Jianda et al., 1997; H. Xue , et al, 2001), fire graphic technology(P Beaudoin, et al,2001) and

application of fire simulation(Wang Jian and Fan Weicheng, 1996; Jurij Modic, 2003). But there are very few

software systems that can simulate the building structural response due to fire with the development process of

building fire. In order to study the dynamic building response with the development of a building fire, a virtual

reality system was developed, which can simulate the building structural fire response synchronously with the

development of a building fire.

A variety of software has been developed in the area of fire simulation, such as FDS, PHOENICS and FLUENT

(Jiang Ling et al., 2009). The structural response under a fire can be calculated and simulated with a series of

software, such as ANSYS, ABAQUS, MSC series software and other well known FEM software.

The system developed by the authors is based on the fire simulation software FDS and the structural analysis

software MARC. The temperature scene is obtained from FDS, the structural fire response analysis results are

obtained from structural analysis by employment of MARC which reads the data file of the temperature scene as

input file to the software MARC. It is difficult to realize the simulation of the building structural fire response

synchronously with the development of a building fire in a virtual reality environment. The authors employed

different screens to display the simulation process of fire and smoke spreading and the deformation of the structure

associated with the fire development. Since the models are different in the simulation of building fire and the

simulation of structural fire response, it is difficult to access the two different models correspondingly.

The solutions to this are introduced in the following sections. The system architecture, the algorithms and the

development solutions are also presented in this paper.

229


2. SYSTEM DESIGN

This system was based on a well known VR software Vega, developed by Microsoft Visual C++ 6.0, in order to

accomplish the synchronous simulation between building fire and structural response. The system shows the process

of the dynamic structural response with the development of a building fire in dual windows as shown in Fig. 1,

while the users can walk through in two scenes. The dual windows can be switched from one window to the other.

For more realistic immersion, the building fire scene or structural fire response results are visualized in the full

screen.

FIG.1:�Interface of system�

FDS is adopted in this paper to simulate the temperature variation, the smoke spread in a building, while

MSC.MARC is adopted to analyze the structural fire response. The structural fire response is analyzed based on the

FDS results. To avoid effect of dimensions of models in different calculated environments, the dimensions of

building model in virtual environment are the same as the dimensions of model in FDS. The building model in

virtual environment is constructed by AutoCAD. In addition, a modelling software Creator, which matches software

Vega, is adopted to create structure models at different time. The results of FDS and MARC are used as input data

for this system. In this system, smoke control, deformation control and synchronous control are important steps for

the accomplishment of synchronous simulation, as shown in Fig. 2.

FIG.2:��ystem flow�

230


3. DEVELOPMENT SOLUTIONS

3.1 Smoke control

In Vega, the visualization of flame and smoke are accomplished by special effect module. The technology of flame

visualization is to render texture in cycles, and the smoke visualization is based on particle system (Wang Jingqiu

and Qian Zhifeng, 2001). In this system, flame and the part of smoke are realized by special effect module in Vega,

but the effect of smoke spreading is realized by gird controlling (Chen Chi et al., 2007).

In Vega, particle system is difficult to control directly. The system divides building into many girds and the smoke

density in each gird is calculated by FDS. In each gird, the effect of smoke is created by the textures rendered in

cycles and the original state is set to be closed. When the smoke density reaches the threshold in some girds, the

smoke state in this gird will be set to open and simulation will show smoke spreads to these gird fields, as shown in

Fig. 3. The general controlling method of smoke spreading in girds as show in Fig. 4, in which the numbers

represent different types of smoke density in girds and the type 4 means the smoke density is beyond threshold.

FIG.3: Smoke controlling flow FIG.4: Smoke controlling schematic diagram

3.2 Deformation control

This system shows the structural response under a fire only by structural deformation, not including the stresses of

structural member. In frame analysis of MARC, structural members are displayed in the form of line and the point

displacements at different time can be calculated. Similarly, this system adopts line to display structural members

and use point displacements to control structural deformation.

Deformation control is to construct structural deformation models at different time by MARC results. In this system,

models at different time are constructed by calling the function from Creator. Structural members are divided into

many small lines and MARC calculates and outputs the positions and displacements for the ends of these small

lines.

According to these positions and displacements, the system calls the function mgSetCoord3d() to draw the lines in

Creator and calls the function mgSetAttList() to set the gradual colour, which shows the value of displacement

(MultiGen-Paradigm Inc, 2001). The whole structural models of building at different time can be constructed in this

method, Fig. 5. The red intensity of the colour is proportional to the value of displacement.

FIG.5: Structural deformation model

231


3.3 Synchronous control in time

Vega applications include two parts: original setting and main loop (MultiGen-Paradigm Inc, 2004). In the main

loop, the 3D scene is rendered in real time. Synchronous simulation between building fire and structural fire

response requires a synchronous rendering of two scenes in the main loop. Two processes must be executed in the

same time-step, so that synchronous simulation of dynamic process can be achieved.

For the simulation of building fire, FDS outputs smoke density in girds in certain time-steps and system identifies

the girds that the smoke has spread to. In the main loop, the smoke states of girds are set as open at every time-step

if smoke density of these girds reach threshold. In this method, the simulation of building fire can be accomplished.

For the structural response under a fire, MARC outputs displacements at the same time-step as building fire and the

structural deformation models at each step are created by these displacements. In the main loop, the structural

deformation models are loaded into the related scene at every time-step. The simulation of structural fire response

therefore can be accomplished.

Although two simulations adopt the same time-step, the simulation is not precisely synchronous, because different

realization methods cost different time in two simulations. So time control is necessary. Function vgGetTime() in

Vega library can be used to get the time from the beginning of the main loop to the present. For two simulations, the

real time spent at each time-step can be gained by vgGetTime(). When the difference of two spent time is beyond

the threshold, fast simulation will pause to wait for slow simulation and keep in same step.

3.4 Synchronous control in space

In this system, synchronous walk-through is an important function. The user location keeps equivalent at the two

scenes in the virtual environment. To realize the synchronous walk-through, the settings of two scenes, objects and

motion models must be consistent and two scenes need to be driven on a same input device.

There are many important parameters in Vega, such as window, channel, scene, and observer. To realize the

synchronous simulation in two screens, these parameters are set in two different ways and two settings need to keep

consistent in each term. The dimension of building model is the same as the structural model and the locations

where two models are loaded in two scenes also keep consistent. In addition, the motion mode at different scenes

must be the same. In the process of simulation, users act as two observers at the same time and observe two

windows, walking in two scenes by one input device, as shown in Fig. 6.

FIG.6: Synchronous walk-through flow

232


4. AN APPLICATION EXAMPLE TO THE SYSTEM

A two-floor house made up of light steel frame is selected for an example for system application. The system

simulates a fire on ground floor caused by sofa in living room and the time of simulation is 15 minutes. Steels have

no any fire prevention measures, covered by a layer of gypsum board. To display the feature of structural

deformation obviously, the values of deformations are enlarged properly.

The synchronous simulation of building fire and structural fire response is showed as follows:

FIG.7:�Structural deformation and fire situation (1 min) Fire is small and few smoke overflows from windows on right wall. There is no deformation caused by fire.

FIG.8:�Structural deformation and fire situation (8 min) Fire grows rapidly and plenty of smoke can be seen in top of firing rooms.

The structural deformation firing rooms get larger and some members with large deformation grow red.

FIG.9:�Structural deformation and fire situation (15 min)

233


Fire burns fiercely. Large number of smoke overflows from all windows and doors and flame can be seen in the window of firing

rooms. There have been severe deformation in some important members and the whole structure is in danger.

The conclusion can be found from Fig. 7 to Fig. 9 that structural deformation grows with development of fire, which

agrees with common sense. In addition, structural deformation concentrate in firing rooms, although smoke has

spread into many rooms. The distribution of deformation is consistent with the distribution of temperature. At 15

minute, the temperature distribution calculated by FDS is show in Fig. 10.

FIG.10:�Temperature distribution in building by FDS

An image from inner of firing room is showed in Fig 11, which helps users study link of building fire and structural

deformation further.

FIG.11:�Fire situation and structural deformation in firing room (10min)�

In firing room, fire is fierce and smoke layer is so thick that wall and ceiling can’t be seen. If simulation just has one

scene of building fire, the structural information can’t be gained. But in synchronous simulation, structural

deformation can be observed clearly in structural deformation scene. Some beams, which are deformed severely, are

red thoroughly.

At this time, severe deformations have taken place in the structure of firing room and the possibility of local failure

is large. Facing this kind of fire situation, fire men should stop entering this room and evacuate from this room in

virtual training.

5. CONCLUSION

The system is developed for the display of fire development process and for the learning of the building structural

response rules. Though it is currently a prototype system it shows the application potential for the training of fire

fighting and the checking of the building fire resistance designs. The results show that the process of dynamic

structure response with the development of a building fire in a visual and lifelike way can be realized by

employment of the solutions presented in this paper.

234


6. REFERENCES

Chen Chi, et al.(2007). "A building fire simulation system based on virtual reality", Journal of Natural Disasters,

Vol.16, No.1, 55-60.

Fu Zhuman , and Fan Weicheng.(1996) . "Building fire simulation method and development", Exploration of Nature

�Vol.15,No.1,28-33.

H. Xue , et al.(2001). "Comparison of different combustion models in enclosure fire simulation", Fire Safety

Journal, Vol.26, No.1, 37-54.

Jurij Modic.(2003). "Fire simulation in road tunnels", Tunnelling and Underground Space Technology, Vol.18,

No.5, 525-530.

Jiang Ling, et al.(2009). "A brief analysis in the current development of fire computer simulation technology", Fire

Science and Technology, Vol.28, No.3, 156-159.

MultiGen-Paradigm Inc. (2004) . "Creating Models for Simulations (version3.0.1) ", U.S.A.

MultiGen-Paradigm Inc. (2001). "Vega Programmer’s Guide (Version 3.7) ", U.S.A.

P Beaudoin, et al.(2001). "Realistic and Controllable Fire Simulation", GI 2001: Graphics Interface, Ottawa,

Canada, June.7-9, 159-166

Wang Jian, and Fan Weicheng.(1996). "Numerical simulation of fire process of multi-rooms", Journal of University

of Science and Technology of China, Vol.26, No.2, 204-209.

Wang Jingqiu, and Qian Zhifeng.(2001). "Fireworks simulation based on particle systems", Journal of Nanjing

University of Aeronautics & Astronautics, Vol.28, No.3, 166-169.

Yao Jianda, et al.(1997). "Applications of FZN model in building fire", Journal of University of Science and

Technology of China�Vol.27, No.3, 304-308.

235


PHYSICS-BASED CRANE MODEL FOR THE SIMULATION OF COOPERATIVE ERECTIONS

Wei Han Hung, PhD. Student,

Computer-Aided Engineering Group, Department of Civil Engineering, National Taiwan University.

[email protected]

Shih Chung Kang, Assistant Professor,

Computer-Aided Engineering Group, Department of Civil Engineering, National Taiwan University.

[email protected]

ABSTRACT: Cooperative erections are often very critical in modern construction projects. The use of

visualization technologies to simulate cooperative erection activities prior to construction can not only help

isolate problematic areas and plan corresponding operation strategies but can also reduce the risk of

unnecessary incidents. A numerical model is required to accurately simulate cooperative erections and the

physical reactions of a crane and its rigging system. In this research, we modelled a crane using three

sub-modules: (1) suspension module, (2) lifted-object module and (3) manipulation module. This model was

implemented and tested in a dual-crane scenario in order to investigate its feasibility. We found that the

proposed physics-based crane model can help deliver a realistic and interactive visualization.

KEYWORDS: Cooperative, erection, physics model, game physics, crane

1. INTRODUCTION

The use of cranes is becoming more versatile and yet challenging in modern construction projects. The tasks

involved are usually very risky and hence require high accuracy (Chi et al., 2007). The rigging objects are

larger, heavier, and come in more varieties than can be lifted by a single conventional crane. Therefore,

cooperative erections have become a trend in modern construction projects. Since large cranes are not always

available and site rental fees are sometimes very high, the cooperative use of standard cranes available on-site is

a cheaper alternative (Ali et al., 2005). Recently, some researchers have utilized a series of computational

methods to solve the path finding problems involved with the operation of cooperative and multiple cranes (Ali

et al., 2005, Chen and Amin, 2007, Kang and Miranda, 2008). However, due to space limitations and the vast

costs of erection activities, it is difficult to verify and test erection strategies using practical experiments.

In the past, many investigators have employed graphical technologies to simulate crane activities. This is a

low-cost and effective method for identifying potential problems before real construction. O’Connor et al.

(1994), Liu (1995), Amatucci et al. (1997), Bernold et al. (1997), and Stone et al. (1999) developed various

simulation and visualization tools for crane operations. Lipman and Reed (2000) used the Virtual Reality

Modelling Language (VRML97) to provide 3D web-based technologies for managing, accessing, and viewing

construction project information. Kamat and Martinez (2005) automatically generated a dynamic 3D animation

of construction processes using discrete-event simulation DES tools. Huang and Gau (2003) designed an

interactive visual simulation on a cluster of desktop computers for mobile crane training. Kang and Miranda

(2009) developed a numerical method required for automatically simulating and visualizing detailed crane

activities.

Recently, some investigators and companies have introduced the use of physics engines to increase the accuracy

of crane simulations. Chi et al. (2007) proposed a physics-based simulation for the manipulation and

cooperation of cranes. Simlog (2007), CMLabs (2007), and the GlobalSim (2007) corporation developed

training simulators with real-time physical behaviours and a realistic rendering of scenes. These works have

demonstrated the effectiveness of integrating graphical technologies and physics engines, thereby increasing the

fidelity of crane simulations.

In this research, we would like to take this progression a step further. We would like to develop a more complete

and systematic physics model for cooperative crane operations. The main purpose of the research is to develop a

generalized physics-based modelling method for simulating cooperative erections. The latest game and physics

engines are used in the system to generate accurate and realistic crane simulations in real time. A suspension

module, lifted-object module, and a manipulation module were developed for simulating cranes, including the

rigging system (the crane), moving platform (mobile crane), lifting object, and also, lifting devices. The

developed model is able to dynamically simulate object attachment and detachment due to the separated

237


modules. An example case of a dual-crane scenario was used to test and investigate the feasibility of the

proposed model and demonstrate the results of the simulation.

2. MODELLING USING MULTYBODY DYNAMICS

We modelled a crane in multibody dynamics to simulate its physical behaviours, that the objects are composed

of interconnected rigid bodies and joints (Erleben et al., 2005). A rigid body, such as a wheel, cabin, hook, or a

lifting object, is called an actor in this research and their motions are constrained by the joints. In order to more

realistically simulate the physical behaviour of crane operations, we introduced springs and dampers into the

joint model. Four types of joints are used in this research: spherical, revolute, prismatic, and distance joints.

Descriptions of the elements of the physics model (actor and joint) and their representative symbols are shown

in Table 1. Detailed equations of multibody dynamics and their solver can be found in literature (Chi, 2007a,

Erleben et al., 2005). The joints are able to be dynamically attached and detached to simulate the linking and

releasing behaviour of rigging objects. Therefore, a continuous erection activity can be simulated.

TABLE. 1: Description of physics model elements and their representative symbols used in this paper (Chi et

al., 2007, NVIDIA, 2008).

Physics model element Symbol Description

Actor A representation of a rigid body; an idealization of a solid body of finite size and is

un-deformable.

Spherical joint

A spherical joint is characterized by the fact that two points, one from each actor, are

always connected to each other. This means that there are three constraints and equality

in the x, y, and z dimensions of the two points.

Revolute joint

A revolute joint removes all but a single rotational degree of freedom from two objects.

The axis along which the two bodies may rotate is specified with a point and a direction

vector.

Prismatic joint

A prismatic joint permits relative translational movement between two bodies along an

axis, but restricts relative rotational movement. It is usually necessary to add joint limits

to prevent the bodies from getting too far from each other along the joint axis.

Distance joint

The distance joint maintains a certain minimum or maximum distance (or both) between

two points attached to a pair of actors. It can be set to be springy in order to behave like a

rubber band.

3. CRANE MODELLING METHOD

We divided the crane model into three parts as shown in Fig. 1: the suspension module, the lifted-object module,

and the manipulation module. The suspension module and the lifted-object module are the most important parts

of the crane model as they can directly influence the simulated physical behaviours. The suspension module

represents the rigging system of the cranes, including the trolley, cable, block, and hook. The lifted-object

module includes hooks, a lifting object, and optional lifting devices. In general, we can classify the lifted-object

module into two different types: lifting with a lifting device and without any lifting device, as shown in Fig. 2

(a) and (b) respectively. The manipulation module includes the crane body, wheels, cabin, and booms etc,

according to the type of cranes we used. In this paper, we focus on modelling a general type of mobile crane to

simulate cooperative crane operations.

238


FIG. 1: The cranes are divided into three parts: the suspension module, the lifted-object module, and the

manipulation module.

FIG. 2: The suspension module and the lifted-object module: (a) lifted-object module with a lifting device; (b)

lifted-object module without any lifting device.

3.1 Suspension module

The suspension module has four actors as shown in Fig. 3: the trolley actor, the cable actor, the block actor, and

the hook actor. The trolley actor is on the top of this module and is the adapter to the manipulation module.

Although the rigging system of the mobile crane doesn’t have a trolley element, we retain this actor in the

suspension module for generalization purposes. We connect the trolley to the boom using a fixed joint in a

mobile crane; however, a prismatic joint is used in tower cranes.

A spherical joint is used to connect the trolley actor with the cable actor to simulate the swing effect of the

suspension system. In general however, we usually use more than one cable to support the block system in

cranes to lift heavy objects. When the block is twisted during the erection process, the multiple cables will force

the block to retain its original rotation along the system axis. To approach the physics behaviour described, we

apply a joint spring to the orientation along the system axis. However, according to Newton’s second law, a

spring system will form a linear simple harmonic oscillator (David H. et al., 2005). In a real case however,

twisting motions of the block will weaken with time due to the energy lost in twisting the cables. To simulate

the energy loss, we apply a joint damper along the system axis in the spherical joint.

239


The connection between the cable and the block is a prismatic joint. The prismatic joint simulates the extension

and the retraction movements of hoisting operations by changing the length of the linear limit during erection.

The hook is attached into a block using a mechanism of 1-DOF in a general crane block. We use a revolute joint

to simulate the mechanism by attaching the block actor and the hook actor. During the erection process, the

lifting object will pull the hook down and increase the static friction between the hook and the block. Due to this

friction, this revolute joint will not be totally free to rotate. Therefore we apply a joint damper into the revolute

joint here to prevent the endless rotating motion and to simulate the realistic physical behaviours of hooks.

Finally, the hook actor will be attached to lifting objects to complete the suspension module.

FIG. 3: Suspension module: (a) Illustration of the suspension module; (b) Physics model of the suspension

module. The trolley actor will be fixed on the boom actor with a fixed joint in mobile cranes, and will be

connected to the boom actor with the prismatic joint in tower crane.

3.2 Lifted-object module

The lifted-object module includes the object to be erected and any optional lifting devices. The lifting device

(which is normally referred to as a spreader) is commonly used for transferring an evenly distributed load to the

crane (Cartmell et al., 1998). Several erection examples without a lifting device and with a lifting device are

shown in Fig. 4 and Fig. 5, respectively. The slings are the linkages between the hook, the lifting devices, and

the lifting objects. We simulate a sling using a distance joint. In Fig. 4, the lifted-object module without lifting

devices can be simulated as two actors directly connected with multiple distance joints. Lifting with lifting

devices can be modelled as an extension of lifting without lifting devices by adding a lifting device actor

between the hook and the object as shown in Fig. 5. The two actors, the hook and object, are indirectly

connected by a lifting device actor with multiple distance joints (slings). By combining the distance joints with

different relative attachment positions, we can realistically simulate the swinging behaviour of lifting and create.

240


FIG. 4: Lifted-object module without lifting devices: (a), (c), and (e) are illustrations of a lifting example with

different slings distributions; (b), (d), and (f) are physics models relative to (a), (c), and (e.)

FIG. 5: Lifted-object module with lifting devices: (a), (c), and (e) are illustrations of a lifting example with

different types of lifting devices; (b), (d), and (f) are the corresponding physics models.

241


3.3 Manipulation module

The manipulation module represents the main body of the crane. This model would be different for different

types of cranes. In previous works, the manipulation module was built using the closed-form forward kinematics

method (Chi et al., 2007). In order to simulate the detailed physical behaviours of crane bodies during the

erection process, we included multibody dynamics. By attaching joints on each actor of cranes, we can simulate

the real mobile behaviours of mobile cranes. Fig. 6 shows a general mobile crane model representation and its

physics model using multibody dynamics is shown in Fig. 7.

FIG. 6: An illustration of a general mobile crane model.

FIG. 7: The manipulation module of a general mobile crane.

4. IMPLEMENTATION

A computer system was developed to implement the proposed model of this paper. By changing the joint limit

and applying joint motors in the virtual environment, cranes can be operated using predefined scripts or by a

user using user interfaces such as a keyboard or a mouse. The developed system was implemented on the latest

graphics engine, Microsoft XNA, a rendering framework which is generally used to develop computer games

and Xbox360 games (Microsoft, 2008). PhysX, a well-known physics engine with a multibody dynamics solver

which uses the position based dynamics approach (Müller, 2007, NVIDIA, 2008), was also integrated in the

system.

242


FIG. 8: The cooperative erection project of the Dalin Oil Refinery.

To demonstrate the developed physics model, a cooperative erection project of the Dalin Oil Refinery in Taiwan

was simulated. The feasibility of the physics model shown in Fig. 8 was investigated. The lifting object of the

project, a vessel, weighed about 45 tons. It was initially lying down and needed to be erected and lifted to the

designated location. A main lifting mobile crane with a spread bar and a tail lifting mobile crane without a

lifting device were planned for the erection. To simulate this scenario, we built two mobile cranes (one

manipulation module and one suspension module each) and one lifted-object module as shown in Fig. 9. The

simulation result is shown in Fig. 10. From time 0 to time 100, the frames show an erection process using

cooperating cranes. Then, the main lifting crane lifted the object on its own to the designated location by

rotating the booms during time 100 to time 200. A smooth and realistic simulation was performed at 40 frames

per second running on a system equipped with an Intel Core2 Duo 2.13GHz processor with 3GB of RAM and a

Geforce 7950 display card.

FIG. 9: The model of the example case (not including the manipulation module of each crane): (a) illustration

of the cooperating cranes; (b) physics model of the cooperating cranes.

243


FIG. 10: The simulation process of the cooperative erection project.

5. CONTRIBUTIONS

This paper presented a physics-based crane model for simulating erection activities using cooperating cranes.

The major contributions are summarized as follows:

• Configurability: The proposed method separated a rigging system into one or more manipulation

modules, lifted-object modules, and suspension modules. These modules are flexible and can be

reassembled to simulate different combinations of cranes. The configurability allows users to

simulate different types of cooperative erections.

• Versatility: The proposed method can not only be applied to cooperative erections but can also be

applied to single crane operations. It can also be used to simulate tower cranes and the rigging

system used on construction sites.

• Portability: The proposed model can be integrated with existing physics engines which support the

joint constraints. Some physics engines, such as PhysX, can even provide hardware acceleration.

This allows future developers to adopt it in other platforms.

• High-fidelity and continuous simulation: The proposed method allows real-time simulations and

visualizations. The physics feedback is computed in real-time based on the user’s inputs. The

physics model can change dynamically. This allows the simulation of the entire erection cycle, in

which lifting devices and objects are dynamically attached and detached.

6. CONCLUSIONS

This paper presented a physics-based modelling method for cooperative cranes. We separated a crane into three

parts, the crane suspension module, lifted-object module, and the manipulation module, to develop a general and

complete crane model. A system was developed to demonstrate and test the feasibility of the proposed model

using an example case of a dual-crane scenario. The system performed a realistic and interactive simulation in

real time. The proposed crane model is extendable, enabling us to model different kinds of cranes and lifting

244


devices. Future work will focus on model verification to increase the reliability of crane models and improve the

proposed crane model.

7. REFERENCES

Amatucci, E., Bostelman, R., Dagalakis, N., and Tsai, T. (1997). Summary of modeling and simulation for

NIST RoboCrane. Proceedings of the 1997 Deneb International Simulation Conference and Technology

Showcase. Detroit.

Bernold, L., Lorenc, S., and Luces, E. (1997). On-line Assistance for Crane Operators. Journal of Computing in

Civil Engineering, 11 (4), 248-59.

Cartmell, M. P., Morrish, L., and Taylor, A. J. (1998). Dynamics of spreader motion in a gantry crane.

Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering

Science, 212 (2), 85-105.

Cheng, Z., and Amin, H. (2007). Collaborative Agent-based System for Multiple Crane Operation. Internetional

Symposium on Automation & Robotics in Construction.

Chi, H. L. (2007a). Physics-Based Simulation of Detailed Erection Activities of Construction Cranes. Master

Thesis.

Chi, H. L., Hung, W. H., and Kang, S. C. (2007). A Physics Based Simulation for Crane Manipulation and

Cooperation. Computing in Civil Engineering Conference. Pittsburgh, U.S.

CM Labs. (2007). Vortex Simulators. Retrieved 6 30, 2009, from Vortex:

http://www.vxsim.com/en/simulators/index.php

David, H., Robert, R., and Jearl, W. (2005). Foundamentals of Physics. Wiley.

Erleben, k., Sporring, J., Henriksen, K., and Dohlmann, H. (2005). Physics Based Animation. Hardcover.

GlobalSim. (2007). GlobalSim. Retrieved 06 30, 2009, from http://www.globalsim.com/

Huang, J. Y., and Gau, C. Y. (2003). Modelling and Designing a Low-cost High-fidelity Mobile Crane

Simulator. International Journal of Human-Computer Studies, 58 (2), 151-176.

Kamat, V. R., and Martinez, J. C. (2005). Dynamic 3D Visualization of Articulated Construction Equipment,

Journal of Computing in Civil Engineering, 19 (4).

Kang, S. C., and Miranda, E. (2008). Computational Methods for Coordinating Multiple Construction Cranes.

Journal of Computing in Civil Engineering, 22 (4), 252-263.

Kang, S. C., and Miranda, E. (2009). Numerical Methods to Simulate and Visualize Detailed Crane Activities.

Computer-aided Civil and Infrastructure Engineering, pp. 169-185.

Lipman, R., and Reed, K. (2000). Using VRML in Construction Industry Applications. Proceedings of

Web3D:VRML 2000 Symposium. Monterey.

Liu, L. (1995). Construction crane operation simulation. Proceedings of second Congress in Computing in Civil

Engineering. Atlanta.

Ali, A. D. M. S., Babu, N. R., and Varghese, K (2005). Collision Free Path Planning of Cooperative Crane

Manipulators Using Genetic Algorithm. Journal of Computing in Civil Engineering, 9 (2), 182-193.

Microsoft. (2008). XNA. Retrieved 7 14, 2009, from MSDN: http://msdn.microsoft.com/en-us/xna/default.aspx

NVIDIA. (2008). NVIDIA PhysX SDK 2.8 - Introduction. Santa Clara, U.S.A.

O'Connor, J., Dhawadkar, P., Varghese, K., and Gatton, T. (1994). Graphical visualization for planning heavy.

Proceedings of theFirst Congress on Computing in Civil Engineering. Washington DC.

Simlog. (2007). Mobile Crane Personal Simulators. Retrieved 6 30, 2009, from

http://www.simlog.com/personal-crane.html

Sivakumar, P., Varghese, K., and Ramesh Babu, N. (2003). Automated Path Planning of Cooperative Crane

Lifts Using Heuristic Search. Journel of Computing in Civil Engineering, 17 (3), 197-207.

245


Stone, W., Reed, K., Chang, P., Pfeffer, L., and Jacoff, A. (1999). NIST Research Toward Construction Site

Integration and Automation. Journal of Aerospace Engineering.

246


INTERACTION BETWEEN SPATIAL AND STRUCTURAL BUILDING DESIGN: A FINITE ELEMENT BASED PROGRAM FOR THE ANALYSIS OF KINEMATICALLY INDETERMINABLE STRUCTURAL TOPOLOGIES

Herm Hofmeyer, Dr.

Associate Professor in Applied Mechanics, Eindhoven University of Technology, Department of Architecture,

Building and Planning, The Netherlands;

[email protected]

Peter Russell, Univ.-Prof. Dipl.-Ing. M.Arch

Professor in Computer Aided Architectural Design (CAAD, Lehr- und Forschungsgebiet Computergestütztes

Planen in der Architektur), RWTH Aachen University, Germany;

[email protected]

ABSTRACT: To understand the spatial and structural building design process and to help designers involved, the

idea of a research engine has been developed: In this engine cyclic transformations take place between spatial and

structural building designs. With this engine, a design process can be studied closely and subjected to improvement,

and designers can be supported. The transformation from spatial to structural design consists of four sub

transformations: (1) from spatial design to structural topology, (2) from structural topology to mechanical model,

(3) from mechanical model to finite element model, and (4) from finite element model to design recommendations.

Because in step (1) architectural elements are transformed into structural elements only, the resulting structural

topologies are not a-priori kinematically determined. Therefore, in step (2), one of the problems to solve is the exact

description of a kinematically indeterminable topology (at which nodes does spatial freedom exists and in which

direction). In this paper a method for this description and its implementation will be presented, developed at RWTH

Aachen University during a stay of the first author. The method starts with assigning specific finite elements, which

define a relationship between element nodal forces and displacements, to each architectural element: Architectural

columns are made by truss-elements, architectural beams with beam-elements, and architectural walls and floors

are transformed into flat shell elements. For each element the local stiffness matrix and local coordinate system are

calculated, whereafter global element stiffness matrices can be derived. These are merged into a system stiffness

matrix, which is subsequently reduced for boundary conditions and reduced elements. Now, the null space of the

stiffness matrix can be calculated, using the technique of singular value decomposition, which yields, after some

further processing, exactly defined modes of spatial freedom related to the structural topology. The method was

implemented for general 3D-cases in C++ and was checked for specific problems, which will also be presented in

the paper. For further transformations in the research engine, the modes of spatial freedom should be suppressed,

thus resulting in a kinematically determined structure that can be subjected to structural (and later architectural)

optimization. In this paper also some suggestions will be made for methods (to be developed), which will make the

design kinematically determined. The methods presented, partly implemented and partly in development, will be part

of the research engine and as such will provide support to spatial and structural building designers and will provide

insight and new developments in the design process of buildings itself.

KEYWORDS: Structural building design, spatial building design, generative design, finite element method,

kinematically indeterminable.

1. INTRODUCTION

Recently, the idea of a research engine has been developed, in which spatial building designs are modified or

transformed into structural building designs and vice versa by means of a cyclic procedure (Hofmeyer, 2007), Fig. 1

on the left. The research engine provides support in two domains. First of all, the transformation or modification

methods ("Trans. selection" in the right part Fig. 1) can be varied and the resulting spatial and structural design

evolution can be followed (by means of the "Measure" in the right part of Fig. 1), resulting in a study of the design

process. Secondly, the actual spatial and structural designs provide the designer with possible new solutions,

resulting in a study of generative design. The concept of the research engine is a general one and can be interpreted

247


in several ways, e.g. as a conventional architectural or structural design technique, as structural optimization, or as

multi-disciplinary optimization (Hofmeyer & Kerstens 2008). Within the research engine, the transformation from a

spatial design 2n-1 to a second structural design 2n consists of four sub transformations: (1) from spatial design to

structural topology, (2) from structural topology to mechanical model, (3) from mechanical model to finite element

model, and (4) from finite element model to design recommendations (Hofmeyer & Bakker 2008) and Fig. 1 on the

right. Note that sub transformation 3 is a single transformation (although two arrows are used in the figure) and

Struct. design 2n-1 is represented both by the preceding mechanical model and the finite element model that follows.

The first sub-transformation is a set of spatial-structural transformation rules that adds geometrical components

(lines and areas, which later will become structural elements like columns, beams, and plates) to a spatial design. It

is important to realise that a set of such geometrical components (a structural topology which represents a future

structural design) is likely to be kinematically undetermined, which means that components may not be connected at

all or only connected such that the topology is not stable: This means only a small imperfection will be enough for

the structural topology to collapse. For some technical reasons, to be discussed, this set of components therefore

cannot be used with finite element model 2 (see Fig. 1 at the right) to analyse stresses and to generate design

recommendations. It should first be changed in a so-called "mechanical model", a kinematically determined (stable)

model with the necessary boundary conditions (i.e. a foundation and loads). This problem cannot be solved easily.

For example a part of it, specifying the exact degrees of freedom as related to the kinematically undetermined state,

is already very difficult as was also noticed by others (e.g. Rottke 1998).

FIG. 1: Research engine on the left and right (grey), sub-transformations on the right (black).

Recently, some ideas on two methods to generate a mechanical model out of a structural topology (sub-

transformation 2) have been presented (Hofmeyer & Bakker 2008). In this paper, a realized part of one of these

methods is presented in-depth: A method to analyze the set of geometrical components in order to know which

degrees of freedom exist and where these degrees of freedom are located within the set of geometrical entities. The

analyzing method uses principles of the finite element method in sub-transformation 2 (shown as finite element

model 1), and should not be mistaken for the finite element model 2 that is used for stress analyses between sub-

transformation 3 and 4, see Fig. 1 on the right. Note that before the finite element models can be applied,

additionally the geometrical components should be redefined such that a conformal partitioned geometry occurs

(Hofmeyer & Gelbal 2009).

2. STATE-OF-THE-ART

Considering the research engine, in the field of Architecture, Engineering, and Construction many research projects

have been carried out to investigate the multi-disciplinary character of the field and to develop computer aided tools

248


to support the design processes involved (e.g. Fenves et al. 1994). In this paper, within the multi-disciplinary design

process, only the disciplines of spatial design and structural design are part of the problem definition. Related

research can be divided in three groups. The first group is descriptive research that develops data models, which

formalize data and their relationships regarding specific aspects of the design process. Related to this paper are data

models that have been specifically developed to relate spatial and structural design (Khemlani et al., 1998, Matthews

et al., 1998, Eastman and Jeng, 1999, Rivard and Fenves, 2000, Scherer and Ghere, 2000, Mora et al., 2006). The

second group is generative research that yields programs, procedures, or concepts for generating spatial and/or

structural design solutions. The oldest but still active field in this group is that of space-allocation that transforms

building requirements into a spatial design (e.g. Kotspoulos 2006). For structural design, a distinction should be

made between research that optimises an existing structural design by means of expert systems, form-finding or

optimization (e.g. Rafiq et al., 2003, Bletzinger and Ramm, 2001, Kocaturk et al. 2003) and research that results in

the actual one-way transformation and evaluation from a spatial to a structural design (Rafiq & MacLeod, 1998,

Fenves et al., 2000). For most research in these two groups, the basic underlying idea is that in the design process a

more or less one-way path runs from spatial to structural design. However, the building design process can also be

modelled with a more cyclic approach, as shown in Fig. 1 on the right (grey). A start is made in cycle n by the

transformation of spatial design 2n-1 into structural design 2n-1, which is often carried out by a structural engineer.

The resulting structural design 2n-1 will be subject to improvement, for example by expert views of other structural

engineers or by optimization techniques. This optimised structural design 2n will be given to the architect and he

will then adjust the spatial design 2n-1 to fit the structural design, which gives spatial design 2n, or to fulfil other

requirements from the building plan yielding spatial design 2n-1 for the next cycle (n increases by 1). The resulting

design cycle -as shown in Fig. 1 on the right (grey)- is defined as "interaction between spatial and structural design"

and the use of this model of the design process in the research engine is justified by many research projects in the

third group, namely on the support of multi-disciplinary design processes, e.g. a building design project can be seen

as a sequence of views and dependencies from several disciplines (Haymaker et al., 2004).

Specifically concerning the problem as defined in this paper, the analysis of kinematically indeterminable structural

topologies, only a very limited amount of information can be found in literature. This is mainly because the use of

the finite element method normally presumes a kinematically determined structure and engineers are educated to

verify whether the structure is kinematically determined before the use of the method. And if their check would not

be rigorously enough, they normally know how to interpret the resulting error signals of the method (extreme

displacements, singular matrix warnings, etc.). Also, a building structure should always be kinematically determined

to be able to be properly used and in the preliminary design phase this is accounted for explicitly. Once the building

structure is then verified by means of the finite element method, kinematically indeterminable structures are not an

issue. What is behind these two explanations is that making a structure (and thus a structural topology) kinematically

determined is a design problem: in general several solutions exist for this and only the structural engineer's

experience and preferences will determine the final design. And because it is a design type problem, it is difficult to

find general applicable methods for support and assistance. Nevertheless, at least for one finite element program (a

user-friendly type and meant for the practical structural engineer) it is known that is helps the structural engineer,

once a kinematically indeterminable structure has been used as input, by suggesting additional structural supports

(MatrixCAE, 2009). This is, however, a limited approach as also more elements (like beams or plates) may be

needed. Finite element programs for more general and academic use (e.g. Ansys Inc., 2007) do not provide such

helpful information. In the field of dynamics, free body motions, which also occur for structures that are defined as

kinematically undetermined in this paper, need not to be a problem if dynamic behaviour is analysed. In several

textbooks this subject is elaborated, including the notion of a matrix's null-space to find the number of free body

modes (e.g. De Kraker and Van Campen, 2001). It can be concluded that procedures for (1) automated analyses of

kinematically indeterminable structural topologies and for (2) automated suggested solutions for a kinematically

determined structural topology are not available in literature and in existing software. Existing literature suggests the

matrix's null-space as helpful in the analysis of kinematically indeterminable structural topologies.

In this paper a matrix's null-space will be used to solve problem (1), the automated analyses of kinematically

indeterminable structural topologies. First, in section 3 the elements (truss, beam, and flat shell) used for the

analysis will be presented, followed in section 4 by a brief introduction of null-space and the algorithms needed to

find it. Then in section 5 a C++ program is presented, as procedure within the partly implemented research engine,

which is used to process some examples shown in section 6. Finally, section 7 and 8 present future methods and

conclusions respectively.

249


3. ELEMENTS

As discussed in section 1 and 2, the finite element method will be a part of the method to analyse kinematically

undeterminable building structures. In this paper no extended review of the finite element method will be given (e.g.

Zienkiewicz and Taylor, 1998) but for the sake of understanding its basic concept will be presented here. In general,

finding displacements, strains and stresses for loaded structures is not possible due to the fact that the displacement

field in the continuous structure is not known exactly. Therefore, the structure is split up in small parts with nodes,

so called finite elements, for which the displacement field within the elements is approximated such that the

relationship between nodal displacements and nodal loads can be determined, for instance by using the principle of

stationary potential energy. Setting equal the displacement of coincident nodes for all elements, some administrative

procedures then yield the system stiffness matrix that relates external loads and nodal displacements. Finally, an

equilibrium equation containing this system stiffness matrix is solved, leading to approximated strains and stresses

for every location in the structure. For the method presented in this paper, three finite element types have been

selected: A beam element, a truss element (derived from the beam element) and a flat shell element.

3.1 Beam element

For the analysing method, a beam element has been selected as presented by Przemieniecki (1968) and shown in

Fig. 2 on the left.

FIG. 2: On the left a beam element as shown in Przemieniecki (1968), on the right a flat shell bending element

developed by Batoz and Tahar (1982).

It is straight and has a uniform cross-section. Two nodes, i and j, and six degrees of freedom (DOF) according to the

local coordinate system, exist for each node. Consequently, each node can resist axial forces, a twisting moment,

two shear forces, and two bending moments. Engineering beam theory can be used to derive the element stiffness

matrix (the relationship between nodal forces and displacements), which means that, given the limitations of beam

theory, the element is not an approximation but yields correct results for the displacements and strains/stresses. The

local element x-axis is defined from node i to node j and the local y-axis is perpendicular to it and in the global x-y-

plane. For the special case that the local x-axis is parallel to the global z-axis, the local y-axis is arbitrarily set equal

to the global y-axis. Finally, the local z-axis is defined by making a right-handed Cartesian local coordinate system.

The beam element has 6 zero energy modes (3 translations and 3 rotations in space) and no spurious modes.

3.2 Truss element

A truss element can only resist axial forces and no shear forces, bending moments, or twisting moments. It can be

derived from the beam element presented in the previous section by removing all terms in the element stiffness

matrix related to the latter mentioned forces and moments. Using Fig. 2 on the left, this means removing possible

250


stiffness terms in the stiffness matrix for all rows and columns except 1 and 7. The truss element has 5 zero energy

modes (3 translations and only 2 rotations in space as rotation along the beam axis is not supported by the node

DOF's) and no spurious modes.

3.3 Flat shell element

For modelling floors and walls, a flat shell element can be used as shown in figure 2 on the right. It is assumed to be

flat and its node numbers should be defined (counter) clockwise. The local coordinate system is defined by the

origin as the intersection of diagonals, the x-axis from the origin to the middle of the line between node j and k, the

y-axis in plane and perpendicular to the x-axis, and finally the z-axis is defined such that a right handed Cartesian

system occurs. Its plane stress behaviour is based on a presentation of such elements in Cook et al. (1989) and its

bending behaviour is based on a so-called DKQ element as developed by Batoz and Tahar (1982). After the authors

repeated the derivations in the last mentioned paper, some typing errors were found in the equations, which are

available via the first author. For both cases (plane stress and bending) numerical integration is applied, 2 * 2 over

the surface. For the drilling stiffness (the ratio between applied force and displacement S6, S12, S18, and S24 in Fig. 2

on the right) 1/1000 of the maximal element stiffness matrix diagonal value is used, as suggested by Kansara (2004).

As a result, the element has 3 zero energy modes regarding its bending part (1 translation and 2 rotations) en 3

modes regarding the in-plane behaviour (2 translations and 1 rotation) and no spurious zero energy modes.

4. NULL SPACE

Once a structural topology (a set of lines and areas) has been translated into a set of beam, truss, and flat shell

elements, for each finite element in this set the local coordinate system and the element stiffness matrix conform this

local coordinate system can be calculated. Then each local element stiffness matrix should be translated into an

element stiffness matrix conform the global coordinate system. This can be carried out using transformation rules as

presented in e.g. Cook (1989). By assuming that for each global node, consisting of all coincident element nodes,

equilibrium exists, and assuming that for all coincident element nodes the same displacements are applicable, the

(system) stiffness matrix for the complete structural topology can be composed. Finally, the system stiffness matrix

is reduced by processing the boundary conditions. Fig. 3 shows a simple structure of two elements and its system

stiffness matrix in global coordinates.

FIG. 3: System stiffness matrix of a simple structure having two truss elements.

The system stiffness matrix relates the forces/moments FSi and displacements/rotations dSi for each degree of

freedom as shown in the figure. Note that for node 1 no degrees of freedom are available because all displacements

are zero (simple support). For node 2, all three degrees of freedom exist, and for node 3, due to two roller supports,

only a displacement in x-direction is possible. If the structure stiffness matrix is singular, i.e. the inverse of the

matrix cannot be determined, the structure is not kinematically determined, in other words: one or more mechanisms

in the structure exist. However, this information is not enough and a method is needed to pinpoint the locations of

the mechanisms in the topology. This method may be based on finding the null space of the structure stiffness

matrix, for which the null space is indicated by a specific set of vectors that satisfy the following condition:

(1)

in which K is the stiffness matrix, and n is a vector of the matrix null space (Kreyszig, 1983). Vectors of the null

space may be summed and may be multiplied with a constant and in all these cases the new vector is part of the null

space of the matrix. Thus null space itself is unique, but its vectors are not. The null space of the matrix of this

system of equations can be determined by converting the matrix in row echelon form U, defining free variables in a

251


new vector x, setting them equal to zero except for one, and solving Ux=0 for each free variable (Hoffman and

Kunze, 1971). Note that it does not matter whether forces do work or not on the structure. Working forces would

only mean that the left term of equilibrium equation in Fig. 3 is non zero, but for the null space only a vector at the

right side that yields the vector at the left side zero is searched for. Vectors of the null space provide information on

which part of the structure forms a mechanism and/or whether the whole structure is able to transform or rotate in

space. This is shown by two vectors that form the null space of the structure in Fig. 3:

(2)

The first vector indicates a free movement of node 2 in y-direction (by a value for S5, the second row in the equation

of Fig. 3) and the second vector shows a free movement of node 2 in z-direction (S6). It should be noted that due to

(large) free movements of node 2 in y and/or z-direction, also node 3 will be able to move (less severe) free in x-

direction. However, this is a second-order effect and thus is not included in the null-space of a (linear) system of

equations. This example suggests the following:

(1) Every kinematically mechanism has a vector in the null space. This can be proved because a mechanism can be

described by a vector, and this vector is part of null space because no forces are needed for the mechanism thus

making the vector at the left side of the equilibrium equation in Fig. 3 equal to zero.

(2) If two mechanisms are connected, the two vectors of null space are still independent, i.e. they do not contain

displacement info for the same nodes. The proof development should take into account the derivation of the

structure stiffness matrix and possibly the algorithm with which the null space is found. As will be shown in section

6, although indeed the number of vectors corresponds to the number of mechanisms, the vectors itself are not

necessarily independent.

5. C++ PROGRAM

It is very laborious to find the structure stiffness matrix and its null space for realistic building related structural

topologies by hand. Therefore a program has been developed to automate this work, also because by using a

program the analysis of kinematically indeterminable structural toplogies can be more easily integrated in the

research engine, which was presented in section 1 and Fig. 1. The program has been written using the programming

language C++ via the Eclipse Platform version 3.4.1, Cygwin version 1.90.4.1, and Microsoft's Windows XP. It uses

input of the geometry of trusses, beams, and flat shells, whereafter for each element the local coordinate system, the

local, and the global element stiffness matrix are derived. The structure stiffness matrix is then composed out of the

element matrices and boundary conditions are incorporated. The matrix's null space is found via singular value

decomposition (Golub and Kahan, 1965), making use of the GNU Scientific Library GSL (GSL Team, 2007).

Finally, the program interprets the null space and outputs the degrees of freedom associated with each null vector,

for example as shown in section 6.1 for truss and beam elements.

6. DEMONSTRATION EXAMPLES

In this section some examples of structural topologies will be presented. They can be used to verify the elements and

program presented and to provide information needed to develop kinematically determined structural topologies.

The latter will be presented in section 7.

6.1 Element verification

The null space of a stiffness matrix not only indicates mechanisms in a structural topology but, because it defines the

singularity of the stiffness matrix, it also shows element formulation problems such as spurious zero energy modes,

which can be interpreted as element deformations not related to nodal forces. In this section 6.1, the elements as

presented in section 3 will be used as input for the program using no boundary conditions and the results can be used

to verify the elements' normal zero energy modes (rigid body movements) and if relevant, spurious zero energy

modes. Furthermore, the output provides information on how to interpret output for more complex cases in the next

sections. For a single truss element (shown in figure 2 on the left, but now without rotational degrees of freedom),

252


positioned parallel to the x-axis, and without boundary conditions, 5 vectors of the matrix null space occur with the

following free mechanisms/movements as output: Free DOF (x) at finite element node 1 (keypoints 1), vector 1, component value: 0.707107

Free DOF (x) at finite element node 2 (keypoints 2), vector 1, component value: 0.707107

Free DOF (z) at finite element node 1 (keypoints 1), vector 2, component value: 1

Free DOF (y) at finite element node 1 (keypoints 1), vector 3, component value: -1

Free DOF (y) at finite element node 2 (keypoints 2), vector 4, component value: 1

Free DOF (z) at finite element node 2 (keypoints 2), vector 5, component value: 1

The first vector indicates free movement of node 1 and node 2 in x-direction, which is correct because in x-direction

node 1 and 2 are connected by the truss. All other vectors indicate movement of an end node in y or z-direction.

Because the element is positioned parallel to the x-axis, these vectors present the most fundamental possibilities for

mechanisms and, as discussed in section 4, combinations of these vectors can be made for more complex free-body

movements. If the element is positioned arbitrarily (not parallel to the Cartesian axes), vectors result that are a

combination of x, y, and z-displacements and consequently these vectors cannot be linked with the rigid body modes

intuitively. No spurious zero-energy modes exist.

For a single beam element, as shown in figure 2 on the left, without boundary conditions, 6 vectors of null space are

listed in the output. Vector 1 to 3 all describe (unequal) translation in the y-direction for both nodes combined with

corresponding (equal) rotations around the z-axis and a free (equal) rotations around the x-axis. Also a very small

translation in the z-direction exists, for which the corresponding rotations (ry) disappear in numerical noise. Vector

4 and 5 represent a transformation in the z-direction for node 1 and 2 respectively with corresponding rotations. Note

that these rotations in radians are very small compared to the translations and as all computer output has some

numerical noise it is important to distinguish correctly between an actual degree of freedom or noise. Finally, vector

6 shows a translation in the x-direction and is equivalent to the truss example vector 1. Six vectors exist,

corresponding to six possible rigid body displacements (x, y, and z-translations and rx, ry, and rz-rotations), but the

output shows that even for elements aligned with the global coordinate system, vectors do not present the most

fundamental possibilities for mechanisms. It seems not to be easy to generate these most fundamental possibilities,

at least making the vector orthogonal is not enough (De Kraker and Van Campen, 2001, Fig. 3.5 and 3.6). No

spurious zero-energy modes exist.

For the flat-shell element a similar situation as for the beam element exists. The output consists of 6 vectors,

corresponding to the six possible rigid body displacements, but not presenting the most fundamental possibilities for

the rigid body modes displacements (x, y, and z-translations and rx, ry, and rz-rotations). No spurious zero-energy

modes exist. However, if the flat shell has a very small thickness, for certain boundary conditions its low bending

stiffness may occur as a spurious zero-energy mode. Now very low values of the diagonal matrix (associated with

the singular value decomposition) should be neglected. It is thus clear that, due to numerical noise, the theoretical

difference between a pure kinematical mechanism and a very low stiffness may be difficult to find.

6.2 Structural topologies

To investigate the output for structural topologies being a concept of real building designs, in this section a specific

structural topology is designed as shown in Fig. 4. It has two levels, to investigate whether a difference in output

exists for two independently unstable levels or the top level being unstable but relative to a stable first level.

Furthermore the elements drawn with dotted lines can be added or left out to investigate the output for rotational

stability, as shown in the table in Fig. 4.

The first model A (72 DOF's), consisting of a stable first level and a complete unstable second level, yields 3 vectors

of the null-space. These vectors represent free displacements in x-direction and in y-direction, and a rotation around

the z-axis. Again, the vectors represent these free displacement modes not in their most fundamental form, but as

combinations of these forms. Also, the rotation around the z-axis is expressed in x and y-displacements of the nodes,

as the rotation around the z-axis of the flat-shell element used is not coupled to the in-plane x and y-displacements.

Model B, having only a rotational unstable second level, shows a single vector of null-space, indicating a rotation

around the z-axis of the upper flat-shell element.

253


Finally model C, totally unstable, yield 6 vectors of the null-space. These are linked to the three x, y, and rz

displacements/rotations of the first level and the three displacements/rotations of the second level. Unfortunably,

every vector contains displacements for all nodes and thus the vectors do not represent the free displacement modes

in their most fundamental form.

FIG. 4: Structural topology (72 DOF's) representing an actual building structure.

7. FUTURE RESEARCH

Now that it is possible for a kinematically indeterminable structural topology to analyse the mechanisms involved,

the next research question is how this structural topology can be made kinematically determined automatically.

Here, some very brief suggestions for the solution of this problem will be presented, which are to be researched in

the near future. One suggested method is to start with a node having free DOF's as close a possible to the ground

supports and being one of the most outer nodes of all other nodes on its height. If its free DOF-components are in x-

direction and there is another node fixed in x-direction at ground level and having the same y-coordinate then a truss

or shell element (the latter having a diagonal equal to the truss element) can be added between the two nodes. For

the y-direction a similar procedure can be carried out. If no additional nodes to the node investigated can be found,

the next node (most outer) on this level could be tried, using the same procedure. An alternative method is to use

also nodes in the "building", which are nodes being not the most outer. In this case they can be connected to nodes

in the same horizontal plane. After each addition of an element, the procedure as presented in section 3 to 6 should

be repeated to study the effect. Only if all nodes at a certain level are fixed in space, the next level can be subject to

the methods. Using one of these two methods (or a combination of both), the structure is made stable from the

ground (supports) up. A third method is to observing the null vectors and if two nodes having the same degree of

freedom (but with different values) and positioned in a (preferably outer) surface parallel to the degree of freedom,

they are coupled. This process is then also repeated until the system is completely determined. Note that using the

suggested methods it will not be uncommon to generate statically overdetermined structures. However, this is not a

problem for further steps like finite element processing.

8. CONCLUSIONS

As part of a so-called research engine, transforming spatial designs to structural designs and vice versa to research

design processes and relationships, a method has been presented to analyse kinematically undeterminable structural

254


topologies. The method has been implemented in a C++ program and is able to use finite truss, beam, and flat shell

elements and it determines the null space of the structure's stiffness matrix. The program have been used to check

that the elements programmed do not show spurious zero energy modes.

A typical building example shows that although the method functions correctly and the number of vectors

corresponds to the number of mechanisms, the vectors of the null-space do not present the mechanisms in the most

fundamental way (i.e. are not independent).

Due to numerical noise, the theoretical difference between a pure kinematical mechanism and a very low stiffness

may be difficult to find.

Given the output of the program, three methods to design a kinematically determined structural topology have been

briefly proposed. They will be researched in the near future.

Parallel to this research is the trend to develop BIM-models which have, in opposition to the past where only data

was included in the set-up, now a more structuring role (Russell and Elger, 2008). This research could be seen as a

related study to the latter role.

9. REFERENCES

Ansys Inc. (2007). Ansys 8.1, Ls-dyna 970 rev 3858, and user manuals, ANSYS, Inc., Southpointe, 275 Technology

Drive, Canonsburg, PA 15317, USA.

Batoz, J.-L. and Tahar, M.B. (1982). Evaluation of a new quadrilateral thin plate bending element, International

journal for numerical methods in engineering, Vol. 18, 1655-1677.

Bletzinger K.U. and Ramm E. (2001). Structural optimization and form finding of light weight structures,

Computers & Structures, Vol. 79, No. 22-25, 2053-2062.

Cook, R.D., Malkus, D.S. and Plesha, M.E. (1989). Concepts and applications of finite element analysis, third

edition, John Wiley & Sons, New York.

De Kraker, B. and Van Campen, D.H. (2001). Mechanical Vibrations, Shaker Publishing, Maastricht, The

Netherlands.

Eastman C. and Jeng T.S. (1999). A database supporting evolutionary product model development for design,

Automation in construction, Vol. 8, No. 3, 305-323.

Fenves S., Flemming U., Hendrickson C., Maher M.L., Quadrel R., Terk M. and Woodbury R. (1994). Concurrent

Computer-Integrated Building Design, PTR Prentice Hall, Englewood Cliffs, New Jersey.

Fenves S.J., Rivard H. and Gomez N. (2000). SEED-Config: a tool for conceptual structural design in a

collaborative building design environment, Artificial Intelligence in Engineering, Vol. 14, 233-247.

Golub, G.H. and Kahan, W. (1965). Calculating the singular values and pseudo-inverse of a matrix, Journal of the

Society for Industrial and Applied Mathematics, Series B, Vol. 2, No. 2, 205–224.

GSL Team (2007). "§13.4 Singular Value Decomposition", GNU Scientific Library, reference manual,

www.gnu.org/software/gsl/.

Haymaker J., Fischer M., Kunz J. and Suter B. (2004). Engineering test cases to motivate the formalization of an aec

project model as a directed acyclic graph of views and dependencies, ITcon, Vol. 9, 419-441.

Hoffman, K.M. and Kunze, R. (1971). Linear algebra (2nd edition), Prentice Hall.

Hofmeyer, H. (2007). Cyclic application of transformations using scales for spatially or structurally determined

design, Automation in Construction, Vol. 16, No. 1, 664-673.

Hofmeyer, H. and Bakker, M.C.M. (2008). Spatial to kinematically determined structural transformations, Advanced

Engineering Informatics, Vol. 22, No. 3, 393-409.

255


Hofmeyer, H. and Gelbal, F. (2009). Redefinition of geometrical components to specify kinematically undetermined

behaviour. Proceedings CIB-W78, 26th International Conference, Management IT in Construction, October

1-3, 2009, Instanbul, Turkey.

Hofmeyer, H. and Kerstens, J.G.M. (2008). A bird's-eye view of cyclic spatial-structural transformations, grammar

based design, and structural optimization, in. (Muylle, M. Ed.) eCAADe 2008, Architecture 'in computro',

Integrating methods and techniques, Proceedings of the 26th Conference on Education and Research in

Computer Aided Architectural Design in Europe, September 17-19, 2008, eCAADe and Artesis, Antwerp,

Belgium, 483-492.

Kansara, K. (2004). Development of membrane, plate and flat shell elements in Java, M.Sc.-thesis, Virginia

Polytechnic Institute and State University, Blacksburg, Virginia.

Khemlani L., Timerman A., Benne B. and Kalay Y.E. (1998). Intelligent representation for computer-aided building

design, Automation in Construction, Vol. 8, No. 1, 49-71.

Kocaturk, T., Van Weeren, C. and Kamerling, M.W. (2003). Integrating Architectural and Structural Form-Finding

Processes for Free-Form Digital Architectures, in (Bontempi, F. Ed.) ISEC-02, System-based Vision for

Strategic and Creative Design, Volume 1, Balkema, Lisse, The Netherlands, 557-562.

Kotsopoulos S.D. (2006). Constructing Design Concepts, A Computational Approach to the Synthesis of

Architectural Form, Massachusetts Institute of Technology (MIT), USA.

Kreyszig, E. (1983). Advanced Engineering Mathematics, Fifth Edition, New York, John Wiley & Sons Inc.

MatrixCAE (2009). MatrixFrame, Wijchenseweg 116, 6538 SX Nijmegen, The Netherlands, +31-24 34 34 380,

www.matrix-software.com/uk/

Matthews K., Duff S. and Corner D. (1998). A model for integrated spatial and structural design of buildings, in

(Sasada T., Yamaguchi S., Morozumi M., Kaga A. and Homma R. Eds.) CAADRIA '98: Proceedings of The

Third Conference on Computer Aided Architectural Design Research in Asia, Osaka University Japan,

Osaka, 123-132.

Mora R., Rivard H. and Bédard C. (2006). Computer representation to support conceptual structural design within a

building architectural context. Journal of Computing in Civil Engineering, Vol. 20, 76-87.

Przemieniecki, J. S. (1968). Theory of Matrix Structural Analysis, McGraw-Hill, New York.

Rafiq M.Y. and MacLeod I.A. (1998). Automatic structural component definition from a spatial geometry model,

Engineering Structures, Vol. 10, No. 1, 37-40.

Rafiq, M.Y., Mathews, J.D. and Bullock, G.N. (2003). Conceptual Building Design-Evolutionary Approach,

Journal of computing in civil engineering, Vol. 17, No. 3, 150-158.

Rivard H. and Fenves S.J. (2000). A representation for conceptual design of buildings, Journal of Computing in

Civil Engineering, Vol. 14, 151-159.

Rottke, E. (1998). ExTraCAD, Computerunterstützung des architektonischen Traagwerkentwurfs, RWTH, Aachen,

Germany.

Russell, P. and Elger, D. (2008). The Meaning of BIM, Towards a Bionic Building, in. (Muylle, M. Ed.) eCAADe

2008, Architecture 'in computro', Integrating methods and techniques, Proceedings of the 26th Conference

on Education and Research in Computer Aided Architectural Design in Europe, September 17-19, 2008,

eCAADe and Artesis, Antwerp, Belgium, 531-536.

Scherer R.J. and Gehre A. (2000). An approach to a knowledge-based design assistant system for conceptual

structural system design, in (Goncalves, Stegier-Garcao, Scherer Eds.) Product and Process Modelling in

Building and Construction, Balkema, Rotterdam.

Zienkiewicz, O.C. and Taylor, R.L. (1998). The Finite Element Method, fourth edition, volume 1, basic formulation

and linear problems, McGraw-Hill book company, London.

256

9th International Conference on Construction Applications of Virtual Reality Nov 56, 2009

VIRTUAL ENVIRONMENT ON THE APPLE IPHONE/IPOD TOUCH

Jason S. Breland

School of Construction, The University of Southern Mississippi

Hattiesburg, MS 39402, USA

[email protected]

Mohd Fairuz Shiratuddin, Ph.D.



[email protected]

ABSTRACT: 3D architecture and construction visualization applications are mostly deployed on desktop, laptop,

and tablet computing devices. Very few 3D visualization applications are developed for mobile devices due to

limitations in computing power and memory. Although there are portable handheld gaming devices that use some

form a 3D game engine to handle 3D graphics quite remarkably, they were never intended for serious 3D

application development.

Recently, mobile devices that can act as a phone or wireless device, a PDA as well as an entertainment system have

been introduced into the consumer market around the world, such as the Apple iPhone, and the Apple iPod Touch.

The rapid change in technology and its affordability has migrated 3D visualization application into these small

ubiquitous mobile devices. Their ability to connect to the Internet wirelessly through Wi-Fi, and/or Third

Generation (3G) network connections, allows for sharing and transferring of digital information among users

regardless of location, at variable speeds. These new mobile devices are also capable of executing 3D applications,

for entertainment and real-time visualization purposes.

Using the Unity game engine for the iPhone/iPod Touch, the authors developed a real-time 3D walkthrough for a

virtual environment application for iPod Touch. In this paper, the authors describe the development process, the two

unique user interaction techniques only available on the iPhone/iPod Touch, the benefits and the challenges, and

finally the potential future improvements to be developed for use in the construction industry.

KEYWORDS: Game engine, mobile, iPhone, iPod Touch, virtual environment

1. INTRODUCTION Two decades ago, the Newton was developed and introduced by Apple Computers as the first personal digital assistant (PDA) device. The Newton and other PDAs in the market were designed for personal use and were not networked for communication purposes. However in recent years, advancements in the communication, Internet and wireless technologies have led to the emanation of smart-mobile devices that combine features of PDAs, mobile phones, and handheld game consoles. The iPhone/iPod Touch is one of the most recent smart-mobile devices introduced by Apple Inc. One main advantage of the iPhone/iPod Touch and similar device is mobility, where users can carry them anywhere. The ability to connect to wireless networks (3G or Wi-Fi) using the iPhone/iPod Touch allows for ubiquitous voice and data communication.

The computing power of the iPhone/iPod Touch not only can handle voice and data communications, it can also handle 2D and 3D handheld quality console games just like the ones running on the Sony Playstation Portable and Nintendo DS/DSi. Having such computing power opens up new possibilities for the iPhone/iPod Touch to be used as an architecture and construction 3D visualization tool for teaching and learning purposes as well as in the field. Devices like the iPhone are now becoming viable tools for offline learning through storage of PDF files for later use or review (Moore, 2009). More and more portable devices like the iPhone/iPod Touch are being assimilated into work environments. This paper outlines the development process of a 3D real-time walkthrough in a virtual environment (VE) application on the iPhone/iPod Touch using the Unity game engine. The authors also describe the development process, the two unique user interaction techniques only available on the iPhone/iPod Touch, the

257


benefits and the challenges of development, and finally the potential future improvements that can be developed for use in the construction industry.

2. IPHONE AND IPOD TOUCH

The iPhone and the iPod Touch were designed, developed, and marketed by Apple Inc. It is the first mobile device that supports multi-touch and tilt-control interaction techniques. The iPhone and iPod Touch have similar functionalities, except the latter does not function as a phone, and does not have built-in camera or microphone. In Table 1, the authors summarize the comparisons between the iPhone and iPod Touch.

2.1 Specifications of the Iphone/Ipod Touch

TABLE 1: A comparison between the iPhone and iPod Touch (Apple, 2009a & 2009b)

iPhone iPod Touch Dimensions &

Weight

Height: 4.5 inches (115.5 mm) Width: 2.4 inches (62.1 mm) Depth: 0.48 inch (12.3 mm) Weight: 4.8 ounces (135 grams)

Height: 4.3 inches (110 mm) Width: 2.4 inches (61.8 mm) Depth: 0.33 inch (8.5 mm) Weight: 4.05 ounces (115 grams)

Display 3.5-inch (diagonal) widescreen Multi-Touch display with a 480-by-320-pixel resolution at 163 pixels per inch

3.5-inch (diagonal) widescreen Multi-Touch display with a 480-by-320-pixel resolution at 163 pixels per inch

Storage Capacity 16GB or 32GB flash drive Holds up to 20 hours, or 40 hours of video

8GB, 16GB, or 32GB flash drive Holds up to 10 hours, 20 hours, or 40 hours of video

Operating System iPhone OS iPhone OS Battery life Internet use:

Up to 5 hours on 3G Up to 9 hours on Wi-Fi Video playback time of up to 10 hours when fully charged Audio playback time of up to 30 hours when fully charged

Internet use: Up to 9 hours on Wi-Fi Music playback time of up to 36 hours when fully charged Video playback time of up to 6 hours when fully charged

Camera, photos,

video recording

Yes Not Available

Microphone Yes Not Available Network

Connectivity

UMTS/HSDPA (850, 1900, 2100 MHz), GSM/EDGE (850, 900, 1800, 1900 MHz), Wi-Fi (802.11b/g), Bluetooth 2.1 + EDR

Wi-Fi (802.11b/g)

The iPhone operating system (OS) is relatively simple compared to the Mac OS X that runs on most Apple computers. The iPhone OS takes up less than 240 Megabytes of the overall storage area of the device. This is considered small compared to the complexity and functionality of the device. One disadvantage of the OS is its inability to run multiple applications at once. For example, a user cannot run an instant messaging application in the background while surfing the Internet. Another drawback is the battery life per full charge. Processing intensive application such as 3D games, continuous data and voice communication to the wireless network, or playing videos, reduce the battery life per full charge.

Apple recently releases the iPhone Software Development Kit (SDK) to encourage development of applications and games from the end-users. This has encouraged end-users to create many innovative and useful applications and

258


games for the iPhone/iPod Touch. Applications and games that are approved by Apple are marketed and sold on the Apple’s “Apps Store” to other users worldwide.

2.2 Limitations of the Iphone/Ipod Touch

According to industrial reviews, the iPhone/iPod Touch lacks some of the features which were expected such as concerning the touch screen. The size of the touch screen is small that it becomes difficult to click on with great accuracy. This perhaps becomes most apparent when the user first tries to use the default keyboard which appears any time texts must be entered. Some users may find that entering little text such as a single sentence is now an arduous task. In addition, the screen will become smeared with fingerprints. Other limitation include, the camera not being able to zoom or record video. There is no simple battery replacement to extend battery life, or in case of malfunction. Another limitation is the network. iPhone users must use the network that Apple has an agreement with in the user’s country. If users do have the data plan, all internet resources are tethered to the device thus not accessible by a laptop or other device (Block, 2008).

Another drawback the authors have found is the inability of the iPhone/iPod Touch to run more than one application at a time. It can be assumed that Apple most likely opted to avoid application multitasking, or tasks running in the background, as a way to preserve battery life and to avoid overtaxing the limited processing capabilities. With one or more active program(s) running in the background, the battery could easily drained in a few hours. The exact processing power of the iPhone and iPod Touch is absent from any known Apple posted information regarding the devices. In fact users had to create applications specifically designed to tax and record resources just to find out the exact processing power of the devices at runtime. Snell (2008) found:

“The iPhone, iPhone 3G, and original iPod Touch all report a 400 MHz CPU and 100 MHz bus. The new

iPod Touch reports a 532 MHz CPU and 133 MHz bus. The physical memory remains the same across all

hardware, at about 117 MB (which Hockenberry interprets, I think correctly, as meaning 128 MB of RAM

with 11 MB being used for video).” (Snell, 2008)

3. GAME ENGINES FOR THE IPHONE/ITOUCH

There are several game engines currently available for the iPhone/iPod Touch. Some are specific only for developing 2D or 3D games, and some support both 2D and 3D games. When Apple first released the iPhone SDK to the public, the only way custom applications or games can be developed was to use the Objective-C programming language in the XCode development environment. Objective-C is the native language for the iPhone/iPod Touch SDK. Developing an application or game using Objective-C can be overwhelming for new developers as all the elements e.g. the Graphical User (GUI), the game-mechanics, physics, lighting, terrain, etc. of the game must be created afresh.

However, with the release of the SDK and large interest from the development community, new game engines for the iPhone/iPod Touch were developed for commercial use and others as freely distributed as Open Source. Table 2 shows some examples of the currently available game engines for development of 3D games on the iPhone/iPod Touch. Unlike game engine for the PC, the number of 3D game engines available for the iPhone/iPod Touch is still limited.

TABLE 2: Examples of 3D game engines for the iPhone/iPod Touch

Game Engine Developer Status Website Link

Sio2Engine

Sio2 Interactive

Open Source http://www.sio2interactive.com/HOME/HOME.html

Oolong

Cannot be determined

Open Source http://www.oolongengine.com

Raydium 3D Cannot be determined

Open Source http://raydium.org/data.php

Ston3D for

iPhone

StoneTrip Commercial http://www.stonetrip.com/shiva/publish-3d-game-on-iphone.html

259


Unity iPhone

Unity Technologies

Commercial http://unity3d.com/unity/features/iphone-publishing

Bork3D

Bork3D Games

Commercial http://bork3d.com

Torque 3D for

the iPhone

GarageGames Commercial http://www.garagegames.com/products/tge/iphone

3.1 The Unity game engine Having tested a few of the available game engine for the iPhone/iPod Touch, the authors selected the Unity iPhone Advanced game engine because of its features, ease of use, faster rate of development and deployment of applications, and its file format support. The Unity game engine is developed by Unity Technologies. Initially, the Unity game engine was only available on the Mac OS X platform for games development with options to deploy the games online, on a Windows PC, or on a Mac OS platform. However, recently Unity Technologies releases Unity 2.5 that allows for development on both Max OS X and Windows platforms. The Unity game engine supports the development and deployment of applications on the iPhone/iPod Touch through the Unity iPhone Publishing (either Basic or Advanced) software.

The Unity and Unity iPhone Advanced development environment is mostly GUI driven. Like most game engines, pre-defined modules already exist in the Unity engines. Modules can be dragged and dropped into a project so that they can be used in a scene. At the very basic level, the modules can be used are they are. However, extended functionalities can be easily added to any of the existing modules through the use of Java scripts or C# programming language (see figure 1). These modules can also be imported and reused in other future projects. Figure xx shows an actual module used in this paper that uses C# programming language. This module sets the screen of the iPhone/iPod Touch to landscape display mode when virtual environment is displayed.

FIG 1: The SetScreen module uses the C# language

4. DEVELOPMENT PROCESS OF A VIRTUAL ENVIRONMENT APPLICATION ON

THE APPLE IPHONE/ITOUCH Most modern 3D games are played in virtual environments (VEs). The main components of the VE are terrain, skybox, lights, 3D assets such as buildings, avatars (to represent player and non-player characters), textures such as the ones applied to the 3D model, terrain, avatars, skybox etc., audio such as sound effects, environmental sound, background music etc., game mechanics and physics. The VE that is created on the iPod Touch consists of a model of a Country House placed on a flat terrain. The game mechanics and physics components of the VE were excluded. Figure 2 shows an overview of the development workflow. Each development stage is described in the following sub-sections.

260


FIG 2: An overview of the development workflow

4.1 Development prerequisites Development of an iPhone/iPod Touch application (or apps) requires the followings:

1. An Intel-based Mac computer. 2. Mac OS X Leopard operating system software installed. 3. An iPhone or iPod Touch hardware. 4. The iPhone SDK available for free from Apple’s website. 5. The Xcode software development tool. Xcode is a suite of tools for developing software on the Mac OS X.

Xcode can be downloaded free of charge from Apple’s website. 6. Join the iPhone Developer Program, and be a licensed Apple developer. A standard program will cost

$99/year. However, by joining the iPhone Developer University Program, Apple will grant free developer’s license to any academic institution that will teach iPhone application development as or part of a course.

(Note: Apple recently updated the iPhone OS to version 3.0 in June 2009. Following the new OS release, the iPhone

SDK was also updated to version 3.0. Prior to using the new iPhone SDK, the Mac OS X operating system must be

upgraded to version 10.5.7, the iTunes software to version 8.2, and Xcode to version 3.1.3.)

Once the authors became Apple certified developers, access to the iPhone Developer Program portal website was granted. Using this portal, a Provisioning Profile and an iPhone Development Certificate were created and installed onto the iPod Touch, to allow for deployment of new locally developed applications on to it. Since the authors are using the Unity iPhone Advanced game engine, the Unity Remote application was installed first on the iPod Touch. The Unity Remote application provides a live preview on the iPod Touch screen by directly linking the Unity Editor environment on the Mac computer to it. This is a useful feature as it allows for quick testing and debugging of the application under development.

One of the development intentions is to use any 3D CAD model developed elsewhere and did not originate from the authors. The reasoning behind this is to test out the level of compatibility of the Unity game engine with respect to 3D CAD model. In this paper, a commercial 3D CAD model of a Country House acquired from BlueBrain3D (2009) was used as part of the VE.

4.2 Terrain

The author separated the development of the 3D CAD model since it was acquired from BlueBrain3D (2009) . The development of the VE for the iPhone/iPod Touch began with the creation of the terrain. Unlike the Unity game engine, the Unity iPhone game engine does not have a dedicated terrain engine; hence terrain can only be created either using geometries created in another 3D modeling software such as 3DS Max or Google SketchUp, and by

Development of VE

261


using a plane geometry (to represent a flat terrain) that is available in the Unity iPhone game engine. In this paper, since the VE depicts a Country House in a suburban area, the use of a flat terrain was sufficient.

4.3 Skybox

Skies were applied using a skybox method. A skybox is a six-sided cube consists of six-seamless images that enclose a virtual environment. Each side of the cube represents North, East, West, South, Top and Down (Shiratuddin et al, 2008, Kanalakis, 2008, Busby et al, 2004). Figure 3 shows the images used for the skybox. A skybox method is usually employed in computer and video games. This method is very efficient since it will make the virtual environment looks larger. Skybox can contain skies, mountains, distant objects such as buildings, and any other objects that are “unreachable” by the user. A skybox can be created using software like Terragen. Another method of creating skies in a virtual environment is to use a Skydome, which is a similar concept to Skybox, but uses a sphere or hemisphere instead. The Unity iPhone Advanced engine supports the rendering of Skyboxes through the use of the built-in RenderFX/Skybox shader.

4.4 Lights and shadows Lights and shadows play an important role in a virtual environment. Without proper lighting and shadows, a virtual environment may look plain and dull. Since the Country House model used in this VE application did not have any interiors, only exterior sunlight was used to generate shadows. A “Directional Light” was used to represent sunlight. A directional light source illuminates a virtual environment using a constant illumination from a single direction. Since a directional light source it is usually dynamic in nature, and it uses a real-time stencil shadow casting method (Shiratuddin et al, 2008). However, in the Unity iPhone Advanced engine, shadows created using directional light sources do not cast real-time shadows; the shadows were baked onto the textures where lights were directed to, using shadow mapping techniques. This gives the illusion of shadow casting. The shadow mapping technique increases frame-rate-per-second (fps) in a VE and prevents the engine from rendering shadows in real-time. This method may not be suitable in virtual environments that require real-time shadows generation e.g. in studies involving determination of shadow intensities and shadow projections in real-time.

With the terrain, skybox and sunlight in place, the VE is now partially assembled in the Unity Editor. Since there is no interaction technique implemented yet, the VE is static.

FIG 3: The skybox images used in the VE

262


4.5 Interaction techniques

Interaction technique is one of the crucial elements in a VE. Some VE applications failed due to improper design and implementation of good interaction techniques. Poorly designed interaction techniques will make it difficult for users to interact with the VE. The iPhone/iPod Touch has unique interaction techniques. The two main interaction techniques are multi-touch and tilt-control movements of the iPhone/iPod Touch itself. For example, in the Google Earth for the iPhone application, by simply tilting the iPhone/iPod Touch up and down, left and right, a user can navigate through the world map. In this paper, the authors implemented both types of interaction techniques for the VE to see which of the two is most suitable for navigation purposes.

Revelle and Reardon (2009) reached several important conclusions when designing learning applications for children in their research. Many of these principles deal with the unique interaction techniques of the iPhone and the problems some of the children had with the interface. Simplifying the game structure was one way to help users from being confused. With reduced screen size and new interaction techniques it is best to keep the game play simple to avoid unnecessary confusion. Using large, visually distinct hotspots is another of their recommendations. The screen needs to be as simple as possible with large buttons so that they can be easily selected. Too many buttons or buttons that are too small quickly become difficult to select without error. Tilt functionality can be tricky for inexperienced users so starting with a low sensitivity is recommended. Users can accidently activate undesired commands if tilt functionality is too sensitive. A setting to adjust sensitivity for advanced users might be useful. The use of audio cues or prompts to describe what actions the user should take can also be helpful. Additional simplified instructions can be applied after 7 to 10 seconds of being idle if the user is still uncertain of what to do. While these are not all of the principles described by Revelle and Reardon (2009), they are some of the most pertinent to navigating a 3D world. These should be considered in the early design phases to avoid problems later.

When first developing the VE, the authors implemented multi-touch interaction techniques (see figure 4). Using this technique, a user simply uses his/her thumbs to navigate in the VE. 2D navigation compass-like images were overlaid on the iPod Touch screen and used as the areas where a user can touch to initiate navigation in the VE. The right compass image once touched, enables a user to look around, and the left compass image allowed forward, backward, left and right movements in the VE. By touching both left and right navigation compass, a user can comfortably navigate in the VE displayed on the iPod Touch’s screen.

FIG 4: The author using the multi-touch interaction technique to navigate through the VE on the iPod Touch

Next, the authors implemented tilt-control movements. Using this method of interaction technique, a user simply tilts the iPod Touch in the direction of navigation to move forward and backward, left and right. This technique however, allows for only navigation in the VE. Looking up and down, left and right could not be combined with this method of interaction, and this posed a navigation problem on the VE. The authors also discovered that if a user tilted the iPod Touch too much, large tilting angle made it hard for the user to see the screen. Due to this reason, the authors concluded that the multi-touch interaction technique is more suitable for navigation in a VE on the iPhone/iPod Touch. Figure 5 shows a snapshot of Unity Editor loading and at the same time running the VE on the iPhone simulator application.

263


FIG 5: The Unity development environment and the iPhone Simulator

5. BENEFITS, CHALLENGES AND CONCLUSION

Some of the benefits of utilizing a 3D game engine, specifically the Unity iPhone Advanced game engine are two- fold. Applications and games can be developed fairly rapidly since the Unity game engine comes with basic modules for real-time 3D applications and games development. These modules are highly customizable through the use of Java scripts or C#. The Unity game engine also has support for major industry standard file formats ranging from 2D images to 3D CAD models. This feature alone is an immense advantage since developers no longer have to be concerned with proprietary non-standard file formats.

The benefits of utilizing the iPhone/iPod Touch for 3D architectural and construction visualization are also clear; the mobility, the sufficient computing power to run 3D real-time applications, the ubiquitous connectivity to wireless networks, and the unique interaction techniques pose a new but rewarding challenge in creating a visualization application. For example with the increased mobility brought on by a handheld device, a client can be briefed about his building over a lunch meeting. Instead of using a laptop, the client can download, visualized, navigate, and walkthrough the 3D model of the building. This type of application can also be used as a review tool on the job site. It could also be used in a small meeting to show the end results of proposed project where a projector or computer may be unavailable or overly cumbersome.

During development, the authors encountered some challenges primarily due to the hardware and software limitations of the iPhone/iPod Touch. Screen size and 3D scene complexity are perhaps two of the largest limitations. The small screen provides excellent picture quality but if the display is cluttered with numerous GUI elements, it can quickly become difficult to click on or begin taking up screen space. The authors have not thoroughly tested what is the 3D scene size limit the iPhone/iPod Touch can display in real-time. However, as with other combination of game engine and hardware specifications, this is an expected limitation.

In conclusion, the authors have succeeded in developing a clean and efficient VE that is easy to navigate and walk through on the iPhone/iPod Touch. The VE application is functional considering the iPhone/iPod Touch size and

Scene assembler window

iPhone Simulator Real-time virtual environment simulator

264


processing power. The authors’ future investigation is to test the exact limitations of the Unity game engine and the iPhone/iPod Touch hardware in detail by comparing frames per second with model complexity and different lighting conditions as well as various navigation techniques unique to the iPhone/iPod Touch. The authors hope to get further opinions and comments from architects, design reviewers, project developers, and educators etc., who may find this type of tool useful in a real world scenario. In short, development for the iPod Touch/iPhone is an ongoing process that the authors’ hope will fully utilize the new interaction techniques and 3D processing power of the devices which make them so unique for development.

6. REFERENCES

Apple (2009a), iPhone – Technical Specifications. Website: http://www.apple.com/iphone/specs.html

Apple (2009b), iPod Touch – Technical Specifications. Website: http://www.apple.com/ipodtouch/specs.html

Apple Developer Connection (2009), iPhone Developer Program. Website: http://developer.apple.com/iphone/program/

Block, Ryan (2008), iPhone 3G review. Website: http://www.engadget.com/2008/07/11/iphone-3g-review/

BlueBrain3D (2009), Bluebrain 3D Model Library. Website: http://www.bluebrain3d.com/3d-models/

Busby, J, Parrish, Z. and VanEenwyk, J. (2004). Mastering Unreal Technology: The Art of Level Design Sams, ISBN-10: 0672326922.

Kanalakis, J. (2008), The Complete Guide To Torque X. AK Peters, Ltd: Wellesley, MA

Moore, J., Oussena, S. and Zhang, P. (2009), A portable document search engine to support off-line mobile learning, IADIS International Conference Mobile Learning 2009, 26th February 2009, Barcelona, Spain.

Revelle, G. and Reardon, E. (2009), Designing and testing mobile interfaces for children, Proceedings of ACM IDC 2009: The 8th International Conference on Interaction Design and Children, pp. 329-332, Como Italy, June 3-5, 2009.

Snell, Jason (2008), That iPod touch runs at 533 MHz. Website: http://www.macworld.com/article/137139/2008/11/ipod_touch_speed.html

Shiratuddin, M.F, Kitchens, K. and Fletcher, D. (2008). Virtual Architecture: Modeling and Creation of Real-Time 3D Interactive Worlds, Lulu.com, ISBN-10: 1435756428.

265


3D VISIBILITY ANALYSIS IN VIRTUAL WORLDS: THE CASE OF SUPERVISOR

Arthur van Bilsen, Ph.D.,

Policy, Organization, Law & Gaming Group



Tel. +31 (0)15 27 87239

[email protected]

Ronald Poelman, M.Sc.,




Tel. + 31 (0)15 27 88542

[email protected]

http://www.tudelft.nl/rpoelman

ABSTRACT: Technology to create virtual representations is getting more advanced every day. The resolution of

areal datasets is increasing and the reconstruction of those sets is more robust and delivers better models. A lot of

those datasets are freely distributed online such as Google Earth. There are cars equipped with stereo

photogrammetric equipment and laser scanners to acquire even more detailed models of reality. But what can we do

with all this information? The datasets can provide us with more than just virtual worlds, object information and

geometric information. In this paper we shift our perspective from the physical objects to the space in between.

From this space a vast amount of visibility data can be obtained, which only recently has become possible in 3D.

Just as one can scan a real physical environment one can also scan a virtual environment. In doing so, we are

analyzing environments by generating 3D isovists in fully 3D models to provide information other than the basic

object and geometric aspects. This information may unlock many possible uses, such as to indicate human activity

and cognitive load in public spaces, to provide security surveillance information and to quantify the diversity of the

living environment. The research suggests ways to use isovists and shows an application thereof in a training

simulation game about safety.

KEYWORDS: Virtual, scanning, visibility, analysis, 3D-isovist.

1. INTRODUCTION

1.1 Virtual environments

There is an increase in use of 3D virtual environments in the architecture, engineering and construction. Many man hours are spend on the modelling processes for the creation of these environments. Although 3D virtual environments are enhancing understanding of the design process, they can be used for other types of understanding as well. These past years, ample attention was directed at graphics. Especially realistic on-line rendering benefitted greatly from this trend. Most noticeable are the developments in the entertainment gaming industry, where vast amounts of new titles appear with appealing visuals. The industry is currently transferring their visual effects from off-line to on-line rendering (Akenine-Möller and Haines, 2002), which means photorealism is brought within reach of large online environments in real-time.

When looking at recent entertainment game titles, it becomes apparent that the complexity embedded in the virtual environments is also increasing. To satisfy the gamers with ever more immersive games, entertainment games display complex weather systems, vegetation systems and artificially intelligent wild-life. And also fire systems are being developed to mimic actual spread of wildfire. These are all systems based on known simulation models that were subsequently stripped, to be able to run in real time. On the visual side, realistic lighting and shadow options are also represented based on high dynamic range lighting, a technique which aims to increase visual realism. A further constraint that is being relieved, is the number of polygons and textures that can be used in a virtual game

267


environment. Together, these developments support simulation of complex systems and their visualisation in greater detail than before.

Not just visualisation is benefitting from technological developments, the process of capturing reality is also making leaps. Reality gets captured and measured more frequently and denser than ever before. The acquisition of areal datasets, which span most cultivated areas on Earth, doubled in resolution the past year. And because of better areal datasets the reconstruction quality is also improving. To name a few, individual lucarnes, windows and chimneys can already be reconstructed from areal laser altimetry and stereo photogrammetry. Parallel to areal-based data acquisition there is an increase in terrestrial acquisition; laser scanning is getting more regularly used and stereo photogrammetry benefits from both computer speed and better sensors. Both laser scanning and stereo photogrammetry systems can be mounted on vehicles to facilitate quick acquisition.

Although it still takes a lot of time to create realistic 3D environments, the process is also becoming more automated due to clever algorithms. Nevertheless, the computer is still not capable of creating the structured detailed environments that feed directly into advanced visualisation engines, without some manual human intervention. For realistic visualisation of environments all kinds of additional information is stored in addition to the basic geometric information. A contemporary entertainment game title has at least 5 layers of texture information attached to its environments and characters. There is a diffuse texture for colour, a light map for the storage of lighting information, displacement and normal maps for additional detail and a specular map for light fall-off. It is these maps and the developments in GPU programming that have made 3D visibility analysis possible. The maps are for example used to store isovist fields, which are shown in the figures of section 3.

1.2 Visibility analysis

Ever since the dawn of sighted life on earth, seeing and being unseen have been key survival skills. Both the hunter and the prey need to take visibility into account as do two potential mating partners. The hawk benefits from an overview, the lion sneaks up as invisible as possible on prey. Other animals, such as those that live in the ground or in trees, construct their hidden habitats based on visibility considerations (Hall, 1966).

Visibility analysis can be seen as a shift in focus from the physical objects to the space that separates them. Where the focus on objects brings efficient ways of storing geometrical and material properties, this shift has other benefits. Space is where people reside, move about and employ activities. Virtual space is where the players, avatars, agents and opponents are, move about and demonstrate their behaviour. While objects may contain information, such as geometrical information, the shift opens up richer ways to investigate the relation between the observer and its spatial environment. This relation is rooted in the survival value, the environment represents. While humans attempt to explore and understand their environment (Kaplan and Kaplan, 1989), visual information is their richest source of information, e.g. in terms of bandwidth, and it is not unlikely that sight has therefore become the dominant sense (for those that have it). Apart from a wealth of colour information, sight also provides proximity information, both of which have obvious survival and reproduction advantages. However, light as a carrier of proximity information, is not the only solution stumbled on by nature. Bats use sound waves in a process called echo-location to generate a kind of depth-array of their surroundings, just like sighted animals (Gibson, 1987). This ‘depth array’ is not only useful for animals, but also for humans.

The process of perception is more than just looking. It also involves walking around objects and touching them. Hence a more complete representation of the 3D environment is formed, than one would expect from a simple directed camera. Apart from peripheral vision, the latter is another argument for the use of the full 360 degrees view, instead of just a narrow forward view. Even though one may be looking forward at one particular moment, in the process of moving and looking around a more complete representation of the environment is usually present in the brain. This representation returns later in section 2.5 as the concept of ‘isovist’, where it aids in formalizing visibility analysis. An isovist is a depth array, or simply what you can see from a single vantage point.

Visibility analysis of man’s habitat: architectural and urban environments, was formalized first by Benedikt (1979). He introduced visual characteristics, called isovist measures, such as visible area, perimeter, variation and skewness. Note that visibility works in two directions. If you have more of your surroundings in view, there are potentially more places from which you can be seen by others. Privacy is an example where this is relevant. It is achieved at observation points with a good view of the surroundings and low angular exposure. In terms of isovist measures this corresponds to high area and skewness (ibid.). Earlier Newman (1973) had already found that it is more likely for

268


crimes to be committed in spaces with poor visibility from the surroundings, as thieves will generally try to remain hidden from sight. From these early observations one finds that visibility analysis can help the design or redesign of buildings and public spaces.

The mathematical framework used in this paper, builds on Benedikt’s framework. In line with Batty (2001, p.124), our main step is not to try and summarize space as do Peponis, Wineman, Bafna, Rashid and Kim (1998a; 1998b), but to work with the full visual field at each vantage point. The mathematical foundation for visibility analyses was laid by Benedikt (1979). It was extended by Van Bilsen (2008) and coined Isovist-based Visibility Analysis (IBVA). However, the actual calculation of visibility measures in practical applications remained a hard problem. Early applications were for the majority conducted on a 2D map and this is still the dominant use today. In the development of three-dimensional measures, first steps have been taken: spatial openness by Fisher-Gewirtzmann and Wagner (2003), sky opening by Teller (2003), real world measurements of the latter by Sarradin (2004) and more recently various measures in digital elevation models (DEMs) by Morello and Ratti (2008). However none of these can be considered a general three-dimensional approach. For example, DEMs cannot adequately represent cities when it comes to observer experience of inside spaces (roofed) and outside spaces (arches, bridges, etc.). Recently, due to the rapid increase in calculation and memory capacity of computer hardware, full three-dimensional analyses have become feasible (Van Bilsen, 2008, 2009). We argue that some of the first results can support the construction of virtual game worlds in new ways and an example from a serious game environment is elaborated on in this paper.

Virtual environments have the potential to fully immerse players, not in the least because of high resolution images, and realistic dynamic lighting, fog, blur and texture effects. The techniques behind these effects can be automated, and are being used by the authors to efficiently analyse virtual environments on visibility characteristics of which an example is given in this paper. We will describe some methods and principles and their relation to visibility analysis.

2. METHODS AND PRINCIPLES

Visibility analysis concepts overlap a great deal with ideas and concepts from related disciplines and vice versa. For clarity, in this section we position visibility analysis in relation to laser scanning, spherical harmonics, ambient occlusion and visibility graph analysis (VGA). We will see that they have many common basic notions. This overlap is an opportunity for synergy among until now mostly separated disciplines. The similarities are remarkable, as for example the main difference between laser scanning and isovist generation is that one is performed in the real world while the other takes place in a virtual world (possibly modelled after the real world). Due to the restrictions of this paper we are not able to provide an exhaustive overview and have selected some with high synergy.

2.1 Laser scanning

A laser scanner is a device that analyzes real-world environments and objects. It collects data on the geometric shape, but allows acquisition of other data, such as reflectance properties and normal direction. There are different laser scanning methods for 3D data acquisition. Most common methods are based on triangulation, time-of-flight and phase-shift. A frustum is captured or even a full sphere with a grid of laser measurements. As laser scanning measurements use light, it is not possible to get information behind the first obstruction a ray of light encounters. Hence it is a line of sight measurement technique. Consequently it generally requires multiple scans to capture a complete object or environment.

The capturing process is very much like the creation of isovists although the main difference is evident; the capturing of reality versus virtual reality. Where laser scans have to deal with atmospheric conditions and limited range, virtual scans have fewer constraints. Although lasers have existed since the 1960s, the use for mid-range laser scanning is relatively new (Fröhlich and Mettenleiter, 2004). To highlight the connection with isovists a type of scanner that most resembles an isovist will be highlighted.

2.1.1 Phase-shift spherical scanners

There is an increasing attention for phase-shift scanners because of high speed and dense scanning capabilities. They can measure more than half a million points per second and provide spherical datasets. The laser rotates vertically with thousands of revolutions per second creating a vertical slice of the scenery in each rotation. Synchronously, it rotates horizontally to change the angle of each slice. The resulting dataset is thus actually a set of vertical slices

269


with the origin being the scanner. These directions also correspond to the isovist’s spherical coordinate directions φ and θ.

Although scanning seems an ideal way to capture the environment, there are some disadvantages which are (partially) shared with isovists. A full dome will approximately take 2 minutes, within that time the condition of the environment might have changed such as lighting conditions or moved objects. In general, virtual environments do not suffer from this limitation as their “time” can in principle be stopped. Obvious exceptions are real-time online environments. The inclination angle of the laser will affect the accuracy; scanners have a footprint which is based on the diameter of the laser beam. Measurements that are more parallel to the laser will have the shape of an ellipse the centre of which is used to get depth information. This naturally affects accuracy: surfaces that are at a 90 degree angle with the laser are not affected by the inclination angle. Furthermore, some colours in the light spectrum will be absorbed or affect the laser negatively, such as dark colours and some red tones. Finally, the range of this type of scanner is quite limited, in general 50-100 meters, due to its set phase-shift principle and eye safety regulations. Isovists can be generated in virtual worlds over much greater distances, limited by floating point constraints.

2.2 Spherical harmonics

Spherical harmonics are used to create realistic real-time lighting in complex 3D scenes. Essentially, the scene’s lighting information is stored in textures of spheres which can be updated when lighting conditions change. Sampling the scene this way is similar to the use of isovists, although the information they store is quite different. Formally, spherical harmonics are the angular portion of a set of solutions to Laplace's equation, represented in a system of spherical coordinates. Laplace's spherical harmonics are a specific set of spherical harmonics which form an orthogonal system.

In 3D computer graphics, spherical harmonics are used for indirect lighting such as global illumination, ambient occlusion and in pre-computed radiance transfer (Green, 2003). Although large amounts of textured polygons can be rendered within a second using state-of-the-art graphics cards, rendering a realistic complex scene at real-time speed remains difficult. With spherical harmonics the rendering primitives are no longer geometrical entities, but sampled images wrapped around spheres. The scene is first sampled from different viewpoints and under different illuminations. Spherical harmonics work at best, and are especially efficient, when illuminated with directional light sources. Whenever the user changes the illumination setting, only a certain number of views have to be reconstructed. Information is stored, such as light direction and light intensity. Although it is a relatively new method to light real-time scenes, it allows image-based objects to be displayed under varying illumination.

2.3 Ambient occlusion

Ambient occlusion is a shading method which adds realism by taking into account the attenuation of light due to occlusion. Ambient occlusion is an approximation process that attempts to simulate the way light radiates in real life. (Langerô and Bülthoff, 2000) It is a global method because the illumination at each point is a function of other geometry in the scene. The effect of ambient occlusion is similar to an object’s appearance on an overcast day which is generally used as a composition layer on 3D scenery. The equivalent of ambient occlusion in the IBVA framework is one minus the sky factor (see FIG. 4).

Because ambient occlusion is using only the geometrical shape of the environments to create its maps it is therefore related to the creation of isovists. Although an isovist calculates a lot more parameters and has full flexibility in the appearance of its parameters there is certainly overlap in the mathematical approach. Ambient occlusion is mostly calculated by casting rays in every direction from the surface. Rays which reach the background or “sky” increase the brightness of the surface but a ray that hits any other object does not contribute to illumination. The result of this type of calculation is that points surrounded by a large amount of geometry are rendered dark and points with little surrounding geometry are light. The maps that are calculated by ambient occlusion are generally stored in the UV space of the textures. It can be visualized as a displacement map or colour texture map.

2.4 Visibility graph analysis

In the early 80s, an approach slightly different from Benedikt’s approach emerged. Hillier coined the representation of urban space as a matrix of the ‘longest and fewest’ lines, the ‘axial map', based on visibility graph analysis (VGA). He also introduced the use of the various versions of the ‘topological’ (i.e. non-metric) measure of patterns

270


of line connectivity called ‘integration’ (Hillier et al, 1982; Steadman, 1983; Hillier & Hanson, 1984). Where IBVA and Benedikt start with continuous space, space is now turned into a visibility graph, which has a discrete nature. This set of analysis methods is known as Space Syntax and one typically uses a 2D map, with polygonal occluders as buildings. Space Syntax has generated not only models for predicting urban movement (Peponis et al, 1989) but has also yielded a practical method for the application of these results and theories to design problems which has a substantial portfolio of projects (Hillier, 1997).

A significant correlation is found between principal lines of sight and pedestrian movement (Hillier, 1998; Carvalho and Batty, 2003). These so-called axial lines can be derived algorithmically from certain isovist-fields (ibid.). It was found (de Arruda Campos, 1997) that the number of axial lines (principal lines-of-sight) intersecting at a square, correlates with the number of people making informal use of the square. Also a correspondence was noted (Conroy, 2001) between junctions, particularly with long lines of sight and large isovist area, and places where people pause on journeys. The approaches of VGA work best in urban environments, where there are many occluding buildings and medium to long streets. However, for more open spaces, the axial lines say less about the behaviour of pedestrians and the method is no longer applicable in these environments. The virtual environment analysed in this paper is rather open and illustrates a case in which VGA is not applicable.

2.5 Isovist-based visibility analysis (IBVA)

This method represents a way of conducting visibility analysis based on the concept of an isovist (Benedikt, 1979). It supports systematical formalisation and application of visibility analysis to real and virtual worlds. As stated in the introduction of visibility analysis (1.2), the first step is not to try and summarize space as do Peponis, Wineman Bafna, Rashid and Kim (1998a; 1998b), but to work with the full visual field at each vantage point Batty (2001, p.124). The mathematical extension towards 3D was described by Van Bilsen (2008). Part of the evolution of visibility analysis as a science, is a proper error analysis for existing 2D analyses (Van Bilsen and Stolk, 2008), since in general the quality of scientific results is strongly related (inversely proportional) to the error made. Although highly focussed on proximity, visibility analysis is broader, as it also includes colour, intensity, information processing and perception of what is seen (Gibson, 1987; Van Bilsen, 2008). Empirical correlations between several measures can be found in the work of Stamps (2005) and Stamps and Krishnan (2004). IBVA is used in the current paper, because a mathematically founded 3D analysis is needed for real and virtual world applications in general, as well as for the Supervisor training environment example below.

FIG. 1 A 2D isovist (blue area, left) and a 3D isovist (volume within blue-beige sheet, right).

From: Van Bilsen and Stolk (2007).

In simple terms, an isovist is “What you can see from a single vantage point” (FIG. 1) as mentioned earlier. More formally, an isovist is a Euclidian, geometrical representation of space (a connected region), as can be seen from one observation point bounded by occluding surfaces. The region of space an isovist encompasses has a volume, an outer boundary area and many geometrical characteristics. In a small room, with curtains closed, one’s isovist volume is small, but out in the open field, one’s isovist volume is enormous, solely bounded by ground, horizon and sky. By calculating visual characteristics, based on the distances to the environment, at a large number of vantage points in space one obtains a scalar field, which is visualized as a heat-map or height-map. The figures in this paper show these planes, called isovist fields, cutting through 3D objects and space.

271


3. SUPERVISOR

A large oil company requested a pilot showing the benefits of serious gaming for the training of supervisors. The resulting training game, Supervisor, can be classified as a serious game. Serious games are simulation games that use technology from entertainment gaming. For creating Supervisor, the Unreal Engine 3 was used (Epic, 2007). In the early stages of the design process, already the decision was made to create an immersive 3D serious game with various scenarios, which need to take place in virtual versions of actually existing drilling sites. Because immersion was considered an important aim of the game, the virtual environment had to at least be realistic. In the game, supervisors are being trained more specifically in health safety and environment (HSE) tasks. One of their tasks is to have a good visual overview of the site. Although this task can be trained by actually walking around the site physically, isovist analysis of the scene can help spot suitable locations on the site, via the virtual version.

FIG. 2 A view inside Supervisor (left) with references to FIG. 9. The model of the virtual world of Supervisor, which

was used for the visibility analysis, stripped of textures for efficiency (right).

Because supervision and in general HSE are of great importance to the client, this case was selected for this paper. Our aims with the visibility analysis of Supervisor’s virtual world are: to assess the degree to which quantitative visibility data provides relevant information for virtual world design. Since Supervisor was purposefully designed for learning about safety, specific attention will be given to safety aspects.

3.1 Isovist-based visibility analysis of the virtual world of Supervisor

In applying IBVA to the virtual world of Supervisor we try to assess the relevance of quantitative 3D visibility analysis for design. The results, however, hint towards further conclusions about the use and possibly the regulations of an environment with regard to safety. Any experimental result is highly influenced by the variables that one chooses to measure. We have not applied a selection criterion on the set of measures available. They are measured in the 3D model and use distance data only. Some may only use the distance data in a horizontal plane at some height (e.g. 160cm) and are called “2D”, keeping in mind the environment is 3D. Other measures use the distances in all directions, covering the entire sphere around the observer, and end with “3D”. The distances are limited to 225m since this includes the complete environment from every observation point. Here is the list of measures that were calculated at each observation point (TABLE 1).

TABLE 1 Long list of measures calculated. Selected measures for this paper have an ‘*’. AREA-2D* The isovist’s area in the horizontal plane AVGR-2D The average distance to the environment, in the horizontal plane AVGR-3D* The average distance to the environment MAXR-2D The maximum distance to the physical environment, in the horizontal plane MAXR-3D The maximum distance to the physical environment MINR-2D* The minimal distance to the physical environment, in the horizontal plane MINR-3D* The minimal distance to the physical environment PER-2D* The isovist’s perimeter, in the horizontal plane SKY-3D* The solid angular fraction of sky visible STD-3D* The standard deviation of the distance to the environment (= )

272


In addition, also the horizontal spatial gradient of these measures was calculated. The mathematical definitions of these measures are taken from Van Bilsen (2008). The measures are calculated on a rectangular spatial grid of about 80,000 observations points slicing through the Supervisor world model of approximately 200m x 100m. The distance between two neighbouring observation points is therefore approximately 0.5 meter, the length of a short step of an adult. The grid resides in a horizontal plane at some height. A plane at 160cm was chosen to observe visibility characteristics. This height holds the middle between average eye-height levels while standing up, kneeled down and bending over. Another plane was chosen at 20cm for comparison.

The isovist generation and isovist measure calculations were done using software from Aisophyst©, according to the following specifications. At each observation point 1,566,726 lines of sight (distances to the environment) are traversed, pointing in all directions. The precision of each distance is floating point: 10-7. At increasing value of the distance, the number of significant decimals will decrease. The minimum distance is 10cm. As the maximum distance is 225m, the error is certainly never larger than 0.5cm, and will typically be lower. Each line of sight covers a concomitant finite solid angle (on average 8·10-6sr) of which the relative error is 1%. Therefore all measures calculated in this paper have a relative error of at most 1%. The data visualisations shown in this paper, and the spatial gradient calculations were obtained with MATLAB™. All isovist measures are calculated in a plane at 160cm unless stated otherwise.

3.1.1 Results

Safety, the supervisor’s overview and equipment

A large average distance to the environment, AVGR-3D, is a good indicator for the amount of accessible volume around an observer. Volume may indicate both available distance as well as visible area (orthogonal to lines of sight) and does not discriminate between the two. Thus it should not be used for detailed results, but may e.g. shed light on the perception of the amount of surrounding transparent volume. Nevertheless, the places where both high AREA-2D and AVGR-3D (FIG. 3) are high are good candidates for a supervisor that needs an overview. At drilling sites, safety equipment such as fire extinguishers should be placed well in sight. And there should always be enough room for fire trucks and ambulances to enter the site. Below, further analyses will be able to provide more detailed information. The black line arbitrarily separates areas where the average distance is higher or lower than 100m. It shows that particularly the larger structures, such as buildings, and not the smaller objects, such as benches, influence available volumetric space.

FIG. 3 The isovist measure AVGR-3D (left) in a 225m sphere. The black line separates places with values below and

above 100m. The isovist measure AREA-2D (right). The black line separates places with high and low value (low is

dark blue, see legends).

In drilling site terminology, the monster spot is chosen at a safe location on the terrain: it needs to have cover, one should be able to count each other, and alternative meeting places should be accessible nearby. In FIG. 3 (left) the monster sport would be chosen outside the black line. In combination with the constraint of being able to stand at least 2 to 3 meters away from objects on the ground (20cm above ground), FIG. 8 (left) tells us there are three candidates for the monster spot: at the top left corner, at the top right, and at the lower middle part of the site.

The area in the horizontal plane indicates the extent to which one has possibilities to see one’s environment in the

horizontal direction at 160cm above the ground: AREA-2D. Within the black lines one has less visible area than outside the black line. Subsequently, outside the black line there is more overview. This measure may also indicate

273


the amount of escape area available, although objects lower or higher remain undetected by this measure and may act as obstacles. This is analysed below by comparison of the 20cm and 160cm isovist fields of MINR-2D.

Safety through good natural lighting conditions and shelter

It takes a small amount of time for the human eye to adapt to sudden changes in lighting conditions. Although it still beats photo and video cameras, within seconds the environment may have changed without notice, such as stacked crates tumbling over (one of the cases designed into Supervisor for players to prevent). The figure highlights places where the natural lighting conditions change suddenly, by showing the gradient of SKY-3D (FIG. 4, left).

FIG. 4 The isovist measure ’gradient of SKY-3D’ (left). High values indicate a change of natural lighting

conditions. The isovist measure SKY-3D (right). The black line encloses places with low lighting (see legend).

Natural light comes from the blue or clouded sky and the sun. The measure SKY-3D (FIG. 4, right) does not discriminate between these, but merely provides the amount of sky visible to a particular observation point, measured in steradians. Nevertheless, it still summarizes the potential of natural light arriving at a point. The black line separates places where more and less than 30 percent of the total sphere around the observer, or in other words 60 percent of the sky’s halfsphere is visible. Within the black lines, the darker areas have poorer natural lighting conditions and are therefore candidates for artificial lighting. As the sky is also the source of various hostile weather conditions such as snow, rain and wind, the measure is also an indicator of the absence of shelter against these conditions. In this manner, the design of both shelter facilities and an efficient lighting distribution may be aided by the SKY-3D isovist field. Note that, the opposite of SKY-3D, for example (1–SKY-3D) signifies the amount of physical environment in view (called ambient occlusion in section 2.3). This can support pursuit of optimal camera placement. Also finding optimal surveillance paths for security guards and law enforcers are among the possibilities.

Safety and information overload

There is a limited amount of visual information that can be processed by the brain in a particular period of time. Early research suggests the bandwidth is about 5 to 9 bits per second (Miller, 1956). The more complex and extensive an environment, the longer it takes to comprehend it. As noted earlier, the process of perception is more than just looking. It also involves walking around objects and touching them. This process may take minutes, hours, days or weeks, depending on the complexity and size of the environment.

FIG. 5 The isovist measures STD-3D (left) and gradient of STD-3D (right).

274


FIG. 6 The isovist measure PER-2D (left) and gradient of PER-2D (right).

An isovist’s perimeter, PER-2D, and the (square root of the) variance, STD-3D, may indicate visual information loads, as these measures are sensitive to fluctuations in the distance to the environment (recall FIG. 1). The latter, STD-3D in FIG. 5 (left), shows a smooth field changing rapidly only when approaching large objects such as buildings. The perimeter (FIG. 6, left), however, provides a more detailed indication of places with high information load. The analysis shows the wealth of depth fluctuation caused by the drilling platform in the middle of the site, which is made of fine-grained steel frames and contains many metal tubes and pipes. Potentially, people visiting the site for the first time, will need time to take in the complexity of the structure (FIG. 2, right), they may stand still suddenly or have less attention for dangerous situations developing in their proximity. From a learning and training perspective, information overload is best avoided, and a more gradual introduction of information is usually preferable. For virtual world designers, these indicators can support finding a suitable information load and hence a desired learning curve.

Safety through a safe 3D distance and safe paths

The safest paths available to a fly at 160cm are not always safest to a walking human, who has to move the entire body from head to toe. The toe may encounter obstructions not present at the head level and vice versa. When choosing as “safest” path the path that leads a pedestrian farthest from obstacles she could collide with, the path coincides with a Voronoi diagram. The lines of this diagram occur where the gradient of MINR-2D is zero in at least one direction. The figures show the differences of the Voronoi diagrams at 20cm and 160cm above the ground. In the Supervisor world, safest paths at 20cm (for your walking feet) are different from the safest paths at 160cm.

FIG. 7 The isovist measure gradient of MINR-2D (Voronoi) at two heights: 20cm (left) and 160cm (right).

Distracted or running people will have to watch the ground in front of them to prevent collisions with low obstructions. The same people will also have to watch their heads, as obstructions may e.g. hang from above. In order to make the differences visible first the minimal distance to the environment is shown at two heights. The majority of obstructions are low, but also some reside higher up. This is shown more clearly by subtracting the two fields: S = MINR-2D160cm – MINR-2D20cm shown in FIG. 9 (left). Places where there is more proximity at 160cm than at 20cm have positive values, of which a subset is painted white. Places where there is more proximity at 20cm than at 160cm have negative values, of which a subset is painted black. The white areas, excluding FIG. 9’s boundary line, indicate danger of tripping over obstacles. The black areas indicate where one is in danger of banging one’s head, such as the stairs shown in FIG. 2 (C).

275


FIG. 8 The isovist measure MINR-2D at two heights: 20cm (left) and 160cm (right).

FIG. 9 The isovist measure S (left) with the cables (A), pipes (B) and stairs (C) shown in FIG. 2, and the gradient of

S (right), where S = MINR-2D160cm – MINR-2D20cm.

Three black areas are places where steel cables for stabilizing the drill tower, are pinned in the ground (FIG. 2, A). A careless pedestrian could easily run into such a cable. The cables are visible in the MINR-2D160cm figure above as paired spots. Nearby white areas indicate the pins used to secure the cables in the ground. The pipe storage in FIG. 2 (B) is identified with a white boundary indicating tripping danger.

The gradient of the subtraction in FIG. 9, (right) shows where the difference between heights occurs most sudden. The figure highlights dangerous spots, where collisions, tripping and bumping may lead to serious accidents. In other words, it shows the intensification of obstructions one can collide with. Alternatively, we could have used the 3D version of MINR at 160cm, which also shows the proximity of obstacles below and above 160cm. From this analysis the site’s layout is most easily identified.

FIG. 10 The isovist measure MINR-3D. The black line marks intersection with a 1 meter proximity sphere.

276


The value of such analyses is found in the support of not only redesign of the physical environment but also for procedures to structure safe use. Although the conclusions are still preliminary, the analyses bring about the potentials and weaknesses of a virtual world, whether it is modelled after a real world or not. Possible design flaws and suggestions for improvement can be calculated and evaluated before a design is realised, shortening the realisation phase and preventing costs of having to correct design flaws afterwards.

4. CONCLUSIONS

The aim of the visibility analysis of (a model of) the virtual world of Supervisor was to assess the degree to which quantitative visibility data provides relevant information for virtual world design. The set of calculated isovist measures, albeit a small selection of all thinkable measures, were connected to safety issues. They were shown to be relevant to real world designers as well as to virtual world designers.

From the isovist fields, determining the best places for a supervisor, i.e. with optimal overview, was possible by combining multiple fields: high AREA-2D and high AVGR-3D. In addition, three suitable locations for the so-called ‘monster spot’ were identified, based on AVGR-3D and MINR-2D20cm. Another practical application was suggested in the placement of safety equipment in spots that need to be well visible. Area’s were identified where lighting and shelter (for weather) conditions are poor or change rapidly by calculating isovist measure SKY-3D and its gradient. The design of an artificial lighting system could be optimized based on this data.

Identifying the spots where information overload may occur, was done based on PER-2D and STD-3D, although it must be kept in mind that these do not calculate information but indicate fluctuations in the depth field, which is a good indicator for information. The analysis shows that the complex metal frames, pipes and tubes of the build structure in the middle of the site, causes a potential increase in information load. Safest paths should be based on proximity measurements (e.g. Voronoi) at heights below a pedestrian’s centre of gravity to prevent tripping or collision. By subtracting isovist fields from two different heights (20cm and 160cm), dangerous places were automatically highlighted white (for tripping) and black (for bumping one’s head). Alternatively, the 3D isovist measure MINR-3D yields a similar result directly, although the direction (e.g. obstacle is low or high) is lost, because this was not stored.

Our future research may develop in several directions. Isovist measures can help explain the behaviour of humans, by quantifying and correlating the characteristics of the space the live in. As a consequence IBVA also supports behaviour modelling of artificial actors in virtual environments, such as agents, opponents, and robots. Building robots can be trained and tested for automated construction behaviour. Artificially intelligent behaviour models can be supported by much more detailed visibility data than before. In any case, the visibility analysis of spatial environment yields a vast amount of data, which may contain support or falsification for existing and future hypotheses. Which hypotheses can be tested remains an open question. Nevertheless, visibility is a general notion and it can work as a connecting concept across soft and hard disciplines.

5. REFERENCES

Akenine-Möller, T. and E. Haines (2002). Real-Time Rendering. Second edition. A. K. Peters Ltd.

Batti, M. (2001). Exploring isovist fields: space and shape in architectural and urban morphology. Environment and

Planning B, Vol. 28, 123–150.

Benedikt, M. L. (1979). To take hold of space: Isovists and isovist fields. Environment and Planning B, Vol. 6, 47–65.

Carvalho, R. and M. Batty (2003). A rigorous definition of axial lines ridges on isovist fields. Casa Working Paper

Series, No. 69.

Conroy, R. (2001). Spatial Navigation in Immersive Virtual Environments. PhD Thesis, Bartlett Faculty of the Built Environment. London, University College London.

de Arruda Campos, M. B. (1997). Strategic Space: patterns of use in public squares of the city of London. First

International Symposium on Space Syntax, London.

Epic Games (2007). Unreal Engine 3. www.epicgames.com.

277


Fisher-Gewirtzmann, D. and I.A. Wagner (2003). Spatial openness as a practical metric for evaluating built-up environments. Environment and Planning B, Vol. 30.

Fröhlich, C. and M. Mettenleiter (2004). Terrestrial laser scanning – new perspectives in 3D surveying. International archives of photogrammetry, remote sensing and spatial information sciences, 8/W2.

Gibson, J. J. (1987). The Ecological Approach to Visual Perception. Lawrence Erlbaum Assoc.

Green, R. (2003). Spherical Harmonic Lighting: The Gritty Details. GDC 2003, San Francisco.

Hall, E. T. (1966). The Hidden Dimension. New York, Doubleday.

Hillier B, (1997). The Hidden Geometry of Deformed Grids: or, why space syntax works when it looks as though it shouldn’t. Space Syntax First International Symposium, London.

Hillier, B. (1998). The Common Language of Space. www.spacesyntax.org.

Kaplan, R. and S. Kaplan (1989). The experience of nature: A Psychological Perspective. Cambridge, Cambridge University Press.

Langerô, M. and H. Bülthoff (2000). Depth discrimination from shading under diffuse lighting. Perception, Vol. 29, 649–660.

Miller, G. A. (1956). The Magical Number Seven, Plus or Minus Two: Some limits on Our Capacity for Processing Information. Psychological Rev., Vol. 63, 81–97.

Morello, E. and C. Ratti (2009). A digital image of the city: 3D isovists in Lynch's urban analysis. Environment and

Planning B. Advance online publication, doi:10.1068/b34144t.

Peponis J, Wineman J, Bafna S, Rashid M, and S.H. Kim (1998a). On the generation of linear representations of spatial configuration. Environment and Planning B, Vol. 25, 559–576.

Peponis J, Wineman J, Bafna S, Rashid M, and S.H. Kim (1998b). Describing plan configuration according to the covisibility of surfaces. Environment and Planning B, Vol. 25, 693–708.

Sarradin, F. (2004). Analyse morphologique des espaces ouverts urbains le long de parcours. École Doctorale, Mécanique, Thermique en Génie Civil. Nantes, Université de Nantes. Doctorat: 224.

Stamps, A. E. I. (2005). Visual permeability, locomotive permeability, and enclosure. Environment and Behavior, Vol. 37, No. 5, 587-619.

Stamps, A.E.I. and V.V. Krishnan (2004). Perceived enclosure of space, angle above observer, and distance to boundary. Perceptual and Motor Skills, Vol. 99, 1187–1192.

Teller, J. (2003). A spherical metric for the field-orientated analysis of complex urban open spaces. Environment

and Planning B, Vol. 30, 339–356.

Van Bilsen, A. (2008). Mathematical Explorations in Urban and Regional Design. PhD Thesis, Delft University of Technology, Netherlands. Available online: search for author’s name ‘Arthur van Bilsen’ in Google.

Van Bilsen, A. and E. H. Stolk (2007). “The Potential of Isovist-Based Visibility Analysis. The Architectural Annual 2005–2006 (Bekkering H. C., Klatte J. and D. Hauptmann, editors), Rotterdam, 010 Publishers.

Van Bilsen, A and E.H. Stolk (2008). “Solving Error Problems in Visibility Analysis for Urban Environments by Shifting From a Discrete to a Continuous Approach.” IEEE Proceedings of ICCSA 2008, June 30–July 3, Perugia, Italy.

Van Bilsen, A. (2009). “How can serious games benefit from 3D visibility analysis?” Proceedings of ISAGA 2009, June 29–July 3, Singapore.

278


EVALUATION OF INVISIBLE HEIGHT FOR LANDSCAPE PRESERVATION USING AUGMENTED REALITY

Nobuyoshi Yabuki, Ph.D., Professor,

Division of Sustainable Energy and Environmental Engineering, Osaka University;

[email protected] and http://www.y-f-lab.jp

Kyoko Miyashita, Graduate Student,


[email protected]

Tomohiro Fukuda, Dr.Eng., Associate Professor,


[email protected]

ABSTRACT: Preserving good landscapes such as historical buildings, temples, shrines, churches, bridges, and

natural sceneries is important. Recently, the number of cases of destroying landscape from view point fields due to

construction of high rise buildings is increasing. To regulate structures such as high rise buildings and high voltage

transmission towers, the government or public agencies have established or are going to constitute height

regulations for building and structures surrounding the landscape target. In order to check whether some portions

of high structures are visible or not behind the target objects from multiple view point fields, it is necessary to make

a 3D model representing geography, existing structures and natural objects using 3D CAD or Virtual Reality (VR)

software. However, it usually takes much time and cost to make such a 3D model. Thus, in this research, we propose

a new method using Augmented Reality (AR). In this method, a number of 3D Computer Graphics (CG) rectangular

objects with a scale are located on the grid of 3D geographical model. And then, the CG models are displayed in an

overlapping manner with the actual landscape from multiple view point fields using the AR technology. The viewing

user measures the maximum invisible height for each rectangular object at a grid point. Using the measured data,

the government or public agencies can establish appropriate height regulation for all surrounding areas of the

target objects. To verify the proposed method, we developed a system deploying AR Toolkit and applied it to the

Convention Center of Osaka University, deemed as a scenic building. We checked the performance of the system

and evaluated the error of the obtained data. In conclusion, the proposed method was evaluated feasible and

effective.

KEYWORDS: Augmented reality, landscape, preservation of landscape, building height regulation

1. INTRODUCTION

Preserving good landscapes such as historical buildings, temples, shrines, churches, bridges, and natural sceneries is

important. Recently, a number of cases showing the destruction of good landscapes from view point fields due to

construction of high rise buildings have been reported. Figure 1 (a) shows a picture of Phoenix Hall of Byodoin, Uji,

Kyoto, which was constructed in 1053 and is a world heritage. Behind the Hall, a tall condominium building can be

seen and it apparently destroys the good landscape. Figure 1 (b) shows a photograph of Yasukuni Shrine, Tokyo,

with a high-rise building behind disturbing the landscape. The reasons such destruction of good landscapes have

occurred are (1) the area of landscape preservation was limited to the near surrounding area of historical buildings,

(2) no regulation existed to the outside of the landscape preservation area, (3) municipal bylaws for regulating

landscapes did not have legal power in Japan. Now that the Landscape Act (MLIT 2004) came into force in 2004, to

prevent landscape destruction and to preserve good landscape, regulation of height of buildings and other structures

both in and outside of the landscape preservation area can be enforced. In order to properly set the height regulation,

it is necessary to figure out the maximum height that does not disturb the landscape from the viewpoint fields for

each surrounding place. It is important to note that the terrain elevations of surrounding area vary from place to

place.

In order to evaluate the maximum height that does not disturb the landscape, called invisible depth, for all the points

behind the specific objects making good landscape, the following four methods have been employed in practice: (1)

279


observe at the site; (2) make a physical model of the area and buildings; (3) interpret aerial photographs; and (4)

make a numerical 3D terrain and building model. The method (1) is easy but not accurate; the method (2) requires

great time and care; the method (3) may be used for the terrain with no buildings and trees; the method (4) can be

used for fixed viewpoints but one cannot evaluate the case when viewpoints are moving.

FIG. 1: (a) Phoenix Hall of Byodoin, Uji, Kyoto. (b) Yasukuni Shrine, Tokyo.

Recently, Virtual Reality (VR) technology is often used for observation and evaluation of landscape by city

planners, designers, engineers, developers, and administrators. VR and 3D urban and natural models allow the user

to explore various landscape scenes from multiple and moving viewpoints. However, if VR is employed in order to

evaluate the invisible height for wide area behind the historical or valuable buildings or structures, one must develop

a detailed and precise 3D city model with existing buildings, trees, and other objects. This could take a long time

and high cost. If such a city model has already been built for other reasons, it can be used without additional cost.

Unless otherwise, making a large 3D VR model may not be a suitable choice just for obtaining the invisible height

in terms of cost-benefit performance.

On the other hand, Augmented Reality (AR) has attracted attention as a technology similar to but different from VR.

AR technology provides a facility to overlap real video images with virtual computer graphics images. This can be

done by showing a special marker to the video camera attached with a head mounted display (HMD) worn by the

user. The marker is linked with a designated object and the system shows the object image on the marker of the

video screen. The similar thing can be done by wearing see-through glasses with a video camera.

The authors perceived that invisible heights from multiple and moving viewpoints can be evaluated using AR

technology without making an expensive 3D VR urban model. Therefore, the objective of this research is to develop

a new methodology for building an invisible height evaluation system to preserve good landscapes using AR

technology.

2. METHODOLOGY

2.1 Overview of the Invisible Height Evaluation Method Using AR

The main idea of the proposed method is when the user observes the landscape object under consideration from the

viewpoint fields, wearing a HMD and a video camera connected to a PC, the AR system displays gridded virtual

vertical scales that show elevations from the ground level and that are located behind the landscape object, on the

HMD with overlapped real video images. The user, then, captures the image and observes the maximum height that

does not disturb the landscape for each virtual vertical scale. This process is iterated for various viewpoints, and

appropriate maximum height for each location behind the landscape object is determined. Then, virtual vertical,

maximum height scale models that should not disturb the landscape are generated and the user confirms whether the

virtual objects are surely invisible, while walking around the viewpoint fields and wearing the AR system.

2.2 Implementation of the Proposed Method

A prototype system was developed for validating the methodology proposed in section 2.1. As for the AR,

ARToolKit (HIT Lab 2009) was used because it is commonly and widely used for AR research in the world. The

280


authors used a standard spec laptop PC, SONY VGN-SZ94PS with RAM of 2.0 GB, VRAM of 256MB, a 1280x800

display, OS of Microsoft Windows XP. A HMD of eMagin, Z800, 3D Visor and a web camera of Logicool QCam

Pro for Notebooks with 1600x1200 pixels were used. The web camera was attached with the HMD.

A marker of the AR system was made for identifying the location and viewing direction of the user. Markers are

usually small, for example, 50x50 mm, for the use of tabletop or desktop AR. However, as the landscape objects are

buildings in this research, the typical size of the virtual, vertical scale is about 300m, and the distance of the scale

from the viewpoint can be up to 5 km, small markers such as 50x50mm may not be visible from the viewpoints and

the numerical errors due to the small size of the marker can be very large. Thus, a marker of which size is

900x900mm was made, as shown in Figure 2.

FIG. 2: Photograph and drawing of the marker.

Virtual vertical scale was developed as an OpenGL computer graphics (CG) object. The shape of each scale is a

rectangular solid which consists of multiple 5m-depth colored layers. Each layer has different color so that the user

can read the height of the scale. In addition, the scale object must be see-through or very thin. Otherwise the scales

would cover the target buildings and the user could not read the maximum invisible elevation for each scale.

3. DEMONSTRATION EXPERIMENT AND RESULTS

To demonstrate the proposed methodology and the developed prototype system, an experiment was executed. First ,

Convention Center and adjacent Gymnastic Hall of Osaka University were selected as an experimental landscape

preservation target because these buildings have highly evaluated property of aesthetic design and no permission

was necessary to perform the experiment. Then, the horizontally flat and open square in front of the center and the

hall was selected as a viewpoint field as shown in Figure 3. The marker was installed at the square.

Then, 50m grid was drawn on the map of Suita Campus, Osaka University as shown in Figure 4. The horizontal axis

was named alphabetically, i.e., a, b, c, etc., and the vertical axis was named in number order, i.e., 1, 2, 3, etc. Each

grid cell was named according to the horizontal and vertical number, e.g., d12, k16, m9, etc. The highest elevation in

each grid cell was measured on the map and was assumed to represent the elevation of the cell. The virtual vertical

scale of rectangular solid was placed so that its bottom elevation is the same as the ground elevation of the cell. This

can be done by measuring the location, including the elevation, of the marker, computing the elevation difference

281


for each cell, and linking the marker and all the scale objects. Table 1 shows the elevation difference between the

marker and all the cells. If all the virtual scales are displayed on the screen, the scale would not be visible or at least,

not readable. Thus, for each time, one row is selected and shown on the screen, and then, the next row is selected

and shown, and so forth.

FIG. 3: Convention center and Gymnastic Hall of Osaka University.

FIG. 4: Gridded map of Suita campus, Osaka University.

282


TABLE 1: Height differences between the location where the marker was placed and grid cells.

FIG. 5: Photographs taken during the experiments at Osaka University.

The experiment was performed by two students (Figure 5). One student wore the HMD and video camera and

looked at the buildings the scales. The other held and operated the AR system and the PC, and captured images.

Sample captured images are shown in Figure 6. From the captured image, the maximum invisible height for each

rectangular solid scale was measured. They also walked around the square and confirmed that it was possible to

view both the real video image and virtual scales, while walking.

Based on the invisible height measured from the captured images, a sample regulation plan was made as shown in

Table 2. Then, all the scales were arranged so that each height was the same as the regulated height and linked to the

marker. The experiment showed that the virtual shortened scales looked shorter than the target buildings from the

viewpoint field (Figure 7).

283


FIG. 6: Captured images of the real buildings and marker with virtual vertical scale rectangular solids.

TABLE 2: A sample regulation plan.

FIG. 7: Captured images of the real buildings and marker with virtual scale rectangular solids of which height are

shortened so that they comply with the proposed regulation plan.

284


4. DISCUSSION

Since ARToolKit is based on the computer vision technique depending on the image of a physical marker on the

video display, errors are born-nature and inevitable. Calibration of camera images must be done and markers should

be carefully and precisely made and installed. Markers should be displayed in a large size relative to the computer

display because the precision depends on the number of pixels representing each edge of the marker. Thus, the

markers should be large enough and should not be placed far from the video camera. Furthermore, the orientation of

video camera relative to the marker and the distance between the marker and the virtual 3D object are important in

terms of the error and preciseness.

The authors executed an experiment to measure the errors prone to the marker orientation and the distance between

the marker and the virtual object. The marker was set at the distance of 7m from the video camera. Four existing real

buildings which are visible from the experiment site and of which precise location and dimension data can be

obtained were selected. Then, virtual 3D wireframe rectangular solid models representing the edges of those

buildings were made using OpenGL and linked to the marker. The distance between the marker and each building

was 124.1 m, 428.2 m, 964.2 m and 2,851.2 m. The orientation from the marker to the video camera varied 0, 15,

30, 45, 60 degrees. A photograph of the site for the case of 2,851.2 m is shown in Figure 8. For each case, the error

of each node of the angle between the actual video image of the existing building and the wireframe virtual CG

model located at the building place in terms of the number of pixels. Figure 9 shows the errors of number of pixels

for the farthest building case (distance = 2,851.2 m). For the orientation of 15, 30, 45 degrees, the errors are less

than 5 pixels, while for the 0 and 60 degree cases, the error was over 10 pixels. Figure 10 shows the relationship

between the average height errors in meter, converted from the pixels, and the distance between the marker and the

existing buildings for 5 different orientation cases. Apparently, the cases of 0 degree, i.e., the video camera was

located just in front of the marker, indicated large errors of over 15m, which suggests the inability. However, for

other cases, including the farthest building, the average errors were less than 7m.

FIG. 8: The experimental site for the case of distance = 2,851.2 m. Points A and B are the nodes of the virtual

building model. The building is 50 story high.

285


FIG. 9: Errors of number of pixels for the farthest building case (distance = 2,851.2 m)

FIG. 10: The relationship between the average height errors in meter, converted from the pixels, and the distance

between the marker and the existing buildings for 5 different orientation cases.

5. CONCLUSION

In order to preserve good landscape, regulation of height of newly designed buildings is necessary. VR technology

and other methods been employed to measure the allowable height for buildings and other structures near the site.

However, those methods have drawbacks. Thus, in this research, the authors proposed a new methodology for

evaluating the invisible height of virtual buildings that may be designed in the future from the multiple viewpoint

fields using AR technology. Then, the prototype system was developed and applied to a sample good landscape site

at Osaka University. To reduce errors, a large marker was built. Experiments for evaluation errors were executed

and error analysis was performed. Based on the maximum invisible height at various locations near the site, a

sample regulation plan was made. The experiments showed the feasibility and practicality of the proposed

methodology. For future work, as the size of the marker is limited, new methods such as using existing large

structures as markers are explored.

6. REFERENCES

HIT Lab (2009). “ARTookKit.” http://www.hitl.washington.edu/artoolkit/

MLIT (2004). “Landscape Act.” http://www.mlit.go.jp/crd/townscape/keikan/pdf/landscapeact.pdf

286


AN EXPERIMENT ON DRIVERS’ ADAPTABILITY TO OTHER-HAND TRAFFIC USING A DRIVING SIMULATOR

Koji Makanae, Prof.,

Miyagi University;

[email protected] http://www.myu.ac.jp/~makanae

Maki Ujiie,

ARRK Corporation

ABSTRACT: Traffic orientation is of two types—left-hand traffic (LHT) and right-hand traffic (RHT). With the

increasing cross-border movement of people and the development of worldwide motor vehicle traffic network, the

difference in traffic orientation becomes a barrier in the international traffic environment. The objective of this

study is to clarify drivers’ adaptability to other-hand traffic (right-hand traffic for Japanese drivers) by conducting

an experiment with a driving simulator, which can easily switch the traffic orientation. According to the results of

the experiment, most of the subjects recorded route mistake in the experiment and feel uncomfortable in the other-

hand traffic orientation, and half the subjects recorded route mistakes in the experiment.

KEYWORDS: Right- and Left-hand traffic, Driver’s Adoptability, driving simulator, worldwide traffic network

1. INTRODUCTION

Vehicles travel on the left or right side of the road. The Convention on Road Traffic agreed upon in Geneva in 1949

stipulates that each contracting state should adopt an uniform travel system but has not defined any standard

international vehicular travel system. At present, 75 countries and regions adopt the left-hand traffic system (LHT)

and 164 the right-hand traffic system (RHT). Several countries have changed the system, mostly from LHT to RHT.

As a result, RHT is now prevalent.

Drivers learn to drive under a particular traffic system adopted in their country. As more people move across the

national border in recent years, opportunities have been increasing of driving under a different traffic system. The

construction of big civil engineering structures such as strait-crossing bridges and tunnels has created global road

networks. Then, safely connecting networks incorporating different traffic systems is a present issue. The means for

removing the barriers created by the difference of traffic system include globally standardizing the traffic system

and enforcing drives operating a vehicle across the border to learn driving for adaptation to a different traffic system.

In either way, drivers need to adapt themselves to varying traffic systems temporarily or permanently. Against the

above background, the authors developed a driving simulator (DS) applicable to varying traffic systems. In this

study, driver's adaptability to different traffic systems is evaluated by having domestic drivers who have accustomed

themselves to LHT drive under RHT using the DS.

2. SIGNIFICANCE OF TESTS USING A DRIVING SIMULATOR

Studies of driver behavior frequently require testing under special environments or conditions. Tests using actual

vehicles under actual conditions involve risks for drivers or surrounding environments in numerous cases. Applying

a driving simulator (DS) to these tests enables the reproduction of a certain point in time under the same

environment for numerous subjects because it ensures the safety of subjects and because it can reproduce the test

environment and control test conditions. These benefits are the reasons why DS have been used for various tests and

training such as the tests of driving burden and the assessment of road design. In order to evaluate how scared the

driver is, Matsui et al. (2002) verified the relationship between the distance between vehicles and driver's scare in

tests using a DS. Tests for evaluating the fatigue of drivers who travel long hours were conducted by Nishida et

al.(2003) and others. Thus, DS have been employed as a useful tool for conducting tests that would be difficult

under an actual environment because of the need to ensure the safety of subjects.

287


DS involve high-speed dynamic image processing with the aid of computer graphics that work in cooperation with

the operation of the steering wheel. DS therefore used to be developed based on costly graphics-oriented work

stations. In recent years, however, DS systems can be built based on personal computers owing to the advancement

of image processing capability of personal computers (ex. FORUM8 UC-Win/Road). In the meantime, DS systems

have been developed for practical application that are equipped with a motion controller capable of representing not

only changes in image and sound but also changes in sense of equilibrium due to acceleration or centrifugal force

(Onuki et.al., 2006).

3. DEVELOPMENT OF A TRAFFIC SYSTEM DRIVING SIMULATOR

3.1 Composition of the driving simulator

The driving simulator (DS) discussed in this study is intended for a comparative study of driver adaptability at the

time of switchover to a different traffic system. The requirements specified in DS development are the capabilities

of switchover to a different traffic system, specification of road structures on an ordinary road or expressway

according to the switchover, and control of the timing of emergence of other vehicles. Providing high-resolution

pictures is, however, not so important. In this study, therefore, a personal computer-based dedicated DS is developed

and tests are conducted.

FIG.1 shows the composition of the DS developed in this study. The DS is composed of a personal computer (with

Microsoft WindowsXP), driving control system (Microsoft Sidewinder Feedback Wheel) and liquid crystal display

projector. The driving control system is connected to the personal computer via a USB port.

FIG. 1: The composition of the DS

Languages used for developing the DS are C# as a basis, OpenGL for drawing and DirectX9.0 for acquiring

operation data from the driving system. Course data can be expressed by XML and DOM (Document Object Model)

is used for data input and output.

3.2 Acquisition of driving data and drawing of the vehicle

The data on the operation of the driving system is acquired using DirectInput. Data on the angle of rotation of the

steering wheel and the degree of acceleration or braking is obtained as integers between 0 and 1000. The data is

converted to the angle of rotation of the steering wheel and acceleration or deceleration to calculate the position of

the vehicle and generate an image. A DS user interface is shown in FIG.2. As the initial settings for the DS,

whether or not the position of the steering wheel is changed, whether or not dynamic images of other vehicles exist

and whether or not the degree of control is displayed are specified, and drawing is carried out accordingly.

288


FIG. 2: The DS user interface

3.3 Acquisition of driving data and drawing of the vehicle

Objects of drawing are the road, accessories to the road, surrounding facilities and oncoming vehicles.

(1) Road

The road object is constructed based on the data on the road alignment composed of straight lines, circular curves

and clothoids; and the road cross section. Based on the object, such objects as crossings, T intersections and

interchanges are also prepared as parts and drawn according to their positions.

(2) Accessories to road

As accessories to the road, traffic signals, road signs and vegetation are specified. Traffic signals are designed to be

controllable according to time. The drawing function is specified so that the positions of traffic signal supports may

be changed whenever the traffic system is switched. Multiple types of road signs are generated including regulatory,

warning and guide signs. The supports are also re-positioned when the traffic system is switched as traffic light

supports. Trees can be arranged as vegetation, which is represented by billboards using two textures.

(3) Surrounding facilities

Multiple types of structures can be arranged along the road. Texture mapping is possible for some structures.

(4) Oncoming vehicles

Oncoming vehicles are represented by texture mapping. They are designed to emerge in relation to the position of

the test vehicle.

3.4 Output of travel data

Vehicular travel data is recorded using XML. The current position (coordinates), angle of rotation of the steering

wheel and the degree of acceleration or braking are recorded and output every 0.5 second.

3.5 Construction of an analysis system

An analysis system is built to analyze the travel data recorded by the DS. Graphs and plan views can be drawn for

the speed, time and tracks based on the travel data of a single or multiple subjects.

3.6 Construction of an analysis system

A course editor is generated to facilitate the arrangement of road objects, accessories and surrounding facilities on

the experimental course (FIG. 3). Data is described using XML.

289


FIG. 3: The course editor

4. ADAPTABILITY EVALUATION TESTS

4.1 Test method

Tests were conducted for 13 subjects with a driver's license for an ordinary motor vehicle who had never driven

overseas. Ten males and three females of an average age of 22.1 participated in the tests. Test courses were prepared

for LHT and RHT. The course was composed mainly of an ordinary road, expressway and interchanges. The round-

trip course had right- and left-turning curves, merging zones and entries into parking space, which are the key points

when comparing the travel under different traffic systems. Subjects practiced driving for several minutes to get

themselves accustomed to the driving system and travelled under the LHT and then RHT systems. Data such as the

travel speed and tracks was recorded. Questionnaires were distributed to subjects to report the points where they felt

scared. FIG.4 shows the model course map and the points where travel records were compared under LHT and RHT

systems.

FIG. 4: The model course map

290


4.2 Test results and discussions

FIG. 5 gives graphs showing the time that the subjects required to complete the round-trip. The subjects required an

average of 602 seconds, or approximately ten minutes, to go around the course under LHT and an average of 639

seconds, or approximately 10 minutes and 30 seconds, under RHT, 30 seconds more than under LHT. Actually,

however, less time was required under RHT than under LHT because the course is longer under RHT by as long as

900 m owing to the location of the entrances to interchanges and travelling the distance at a speed of 50 km/hr, the

speed limit specified for ordinary roads, took nearly one minute. FIG. 6 shows changes in speed for subject A.

Subjects travelled at a speed of 71 km/hr under LHT and 75 km/hr under RHT. In response to the questionnaire

distributed after the test, subjects reported that they got accustomed to simulation under RHT because they first

travelled on the left side of the road. The time they spent for practice before the test may have influenced the result.

Upper (red): LHT, Lower (Blue): RHT

FIG. 5: The time to complete the round-trip

Upper (red): LHT, Lower (Blue): RHT

FIG. 6: Changes in speed for subject A

Table 1 gives the number of cases where the subjects explicitly deviated from the lane as indicated by the tracks.

Table 2 shows the points where the subjects felt scared based on the results of the questionnaire survey. The number

of cases of lane deviation under RHT was three to four at intersections (1) and (4). Subjects passed intersection (1),

the first intersection to pass under RHT, while they were not yet accustomed to travelling on the right side of the

road. They therefore may have taken the wrong travel lane (FIG. 7). The cases of lane deviation increased at the

passage of intersection (4) because it was designed that oncoming vehicles would emerge at the intersection.

Subjects probably took the wrong lane while being distracted by the oncoming vehicles. Actually, a vehicle running

291


on the right lane crashed into another. The subject focused attention on one side and may have been careless about

the right-hand traffic. Intersections scored high in the questionnaire survey as a point where the subjects felt

particularly scared.

Table 1: The number of cases where the subjects deviated from the lane

Table 2: The scared points from the questionnaire survey

� (a) LHT� � � � � � � � (b) RHT

FIG.7 : The trajectories at the intersection(1)

� (a) LHT� � � � (b) RHT

FIG. 8: The trajectories at the entrance of ramp way

292


Six subjects out of 14 entered the expressway without noticing the different way of installing the entrance at the

interchange (FIG.8). Under a different traffic system, the merging movements of vehicles on an expressway are

reversed. Subjects may have taken the same path as under LHT. Numerous subjects pointed out in response to the

post-testing questionnaire that particular attention should be paid.

The conclusion is that improper driving frequently occurs at intersections (during left-turning movements in

particular) and at entrances at the interchange when LHT is switched to RHT. These points are therefore considered

risk points. Improper driving is ascribable to distractions by other phenomena in such cases where a traffic system

was switched to another or other vehicles emerged after the passage of an intersection.

5. CLOSING REMARK

In this study, the adaptability to RHT of local drivers accustomed to LHT was evaluated, using a dedicated driving

simulator in order to evaluate the adaptability of drivers to varying traffic systems. Numerous subjects actually

drove improperly under unfamiliar RHT traffic system, running off the road or committing other errors. The post-

testing questionnaire survey revealed that many subjects felt scared under RHT. It was also shown that numerous

driver errors occurred when turning at the intersection or entering the interchange and that the emergence of other

vehicles including those coming toward the subject caused errors.

In this study, analysis was made based on the behavior of a test vehicle reproduced using a driving simulator and the

results of a questionnaire survey. In the future, driver anxiety or sense of fear needs to be evaluated based on more

objective parameters through physiological measurement in such terms as the pulse rate and blood pressure.

Increasing the duration of testing is also necessary to determine the driving experience required for adapting to a

different traffic system. In this study, driving under LHT was simply compared with driving under RHT and a

personal-computer-based simple driving simulation system was developed and applied. In order to increase test

accuracy, it is necessary to conduct tests to evaluate driver adaptability using a driving simulator with motion

control capability that generates a greater sense of reality.

In addition to evaluating driver behavior in testing, studies on social systems are also important. As described in

Section 1, the development of global road networks and the growth of international exchanges are likely to make the

difference in traffic system a barrier in transportation. The two traffic systems established separately in different

countries in the 19th century through the beginning of the 20th century remain, dividing the world. Whether the

switchover of traffic systems and different road networks is possible or not under the present mature transportation

environment is a big research issue in the future. Large-scale social experiments may be required. We should

energetically make efforts to tackle these issues in the future.

6. REFERENCES

Forum8 (2009). “UC-Win/Road”, http://www.forum8.com .

Matsui,Y., Kim, J, Hayakawa,S., Suzuki,T., Okuma,S., Tsuchida,N.(2002). “Evaluation of Fearfulness of Driver

Using Three-Dimensional Driving Simulator”, Proceedings of the Human Interface Sympsium 2002, Sapporo,

Japan, Sep. 1-3, 2002, 131-132. (in Japanese)

Nishida, Y., Shirai, Y., Otsubo, T.(2003). “Development of a driving simulator program for evaluation of long-time

drive tiredness.”, Proc. of Traffic Engineering Meeting(JSTE), Vol.23, 97-100. (in Japanese)

M. Onuki, Y. Suda, Y. Takahashi, H. Komine and K. Matsushita, Study on Improving Angular Acceleration Feel of

the Driving Simulator and its Effects on Drivers’ Behaviors, International Journal of ITS Research, Vol. 4, No. 1,

(2006), 47-52.

293


C2B: AUGMENTED REALITY ON THE CONSTRUCTION SITE

Léon van Berlo, M.Sc.,

Netherlands organization for applied scientific research TNO;

[email protected]

Kristian A. Helmholt, M.Sc.,


[email protected]

Wytze Hoekstra, B. Eng.,


[email protected]

ABSTRACT: This paper describes the development of a system that brings augmented reality at the construction

site. Due to the growing complexity of buildings, the growing number of actors on the construction site, and the

increasing time pressure to construct a building it keeps getting harder to gain a clear overview of the work in

progress. This leads to high costs due to construction failures and even security risks of the building and on the

construction site. To cope with these problems it is necessary to develop a tool that helps construction workers to

rapidly gain inside in the (intended) construction and construction site. The C2B (pronounced ‘see to be’) system

combines the real world with the virtual design giving a mixed reality (augmented reality) view at the construction

site. The C2B system combines technology from different industries to help the construction industry. This paper

describes the development of a prototype of this system.

KEYWORDS: virtual, augmented, reality, world, construction.

1. INTRODUCTION

The C2B (pronounced ‘see to be’, future artists impression above) prototype system provides augmented reality on

the construction site. It was designed and constructed during a project that ran within one of the research programs

at TNO called ‘Mixed Reality and Collaboration’. In this project TNO tried to establish how construction processes

at a construction site could be improved by applying new and innovative technology in the area of mixed reality.

295


1.1 Need for augmented reality

As we have entered the 21st century buildings tend to become much more complex. Through the introduction of

building management systems with electro-mechanical systems buildings seem to be almost changing into complex

organisms. Due to specialization of the construction workers more and a diverse set of actors is walking around on

the construction site. Add to that the increasing time pressure to construct a building and it becomes clear that it

keeps getting more difficult to gain a clear overview of the work in progress for each of the actors involved. This in

turn leads to higher costs due to construction failures. Also security risks of the building and on the construction site

will increase.

To cope with these problems described above it is necessary to support construction workers by providing them with

real-time and accurate information about their ever changing surroundings and the task at hand. We want to do so

without overloading them with information, but helping them navigate through the complex environment they have

to work in. This is were technology for augmented reality (sometimes also known as mixed reality) kicks in. By

adding an extra information layer to the real physical world we can provide the right information in situ at the right

time. There are many different ways of how to ‘augment’ reality. In this article we assume the use of ‘visual

augmentation’. We add extra information into a persons visual view of the world. We see three major categories of

applications using this kind of technology:

• in situ experience. When a building is to be constructed it will alter the landscape. To experience this

feeling people could walk around in that landscape and look at a virtually constructed building and see

how it fits in with the surroundings. What is the impact of a new office building, house or bridge in

the landscape? Early in a project an architect would want to show his views and ideas to other people.

Also, by using environmental models it could be possible to add more abstract information into the

augmented view, like environmental noise contours surrounding the building. The combination of a

real-life environment with the virtual environment can be very enlightening;

• in situ verification. In order to carry out inspections of the construction site a 3D construction map

could be projected on top of a building situation. An inspector could then visually check if the

intended design is in line with how the work was actually carried out. For example the correct

placement of reference poles for brick-laying. This does require a great accuracy; and

• in situ warning. When unseen dangers are present, workers can be warned of these dangers in a more

interruptive and attention demanding way than a sign on the wall (with probably outdated

information).

In this project we wanted to find out to what extent we could engineer a system that would support those

applications using on the market components (with some integration technology of our own). To that, we wanted to

get an idea about the level of ‘applicability’: is it ready for the construction site, or is it an idea for the (near-by)

future?

1.2 Existing approaches

In order to provide people with a view on an augmented reality, several ways already exist. One of the oldest ways is

to draw a picture of reality as you perceive it and add the extra information. This method is time-consuming and

requires a lot of human labor and a lot of craft when the viewers want a realistic view. This method is rather static

too. Not surprisingly photo montage has entered the stage where artists blend in a building in front of other building.

In the digital age a tool like Photoshop in combination with a CAD system is often used. With enough patience and

skill a good impression can be delivered. However, this is not that immersive. You look at a picture which does not

surround you. In order to get an immersive experience there is the Cave Automatic Virtual Environment (CAVE). It

has been there for more than a decade by now. It is an immersive virtual reality environment where projectors are

directed to three, four, five or six of the walls of a room-sized cube. The use of a CAVE requires that you – just as

with the photo montage – first capture reality and bring it into to the cave. This is still a rather static experience.

However, you can move around in a CAVE, by telling the system (using some kind of device) that you want to

move around. The CAVE computes what you should see again and projects it on the screen. Although more real,

you are not actually at the construction site. You do not hear it, feel it and smell it. And since you cannot bring a

CAVE to the construction site, the possibilities end here.

296


Due to the arrival of portable wearable computational and display technology however there is another way of

providing an immersive experience in situ. An important part of this technology is known as the hands-free Helmet

(or sometimes Head) Mounted Display (HMD). It can project a 3D scenery on the retina of the viewer, combining

this with a real-time view of the surroundings. It is a one-person device only and requires a computational device for

computation. Together with today’s portable computer power including powerful 3D graphic cards an in situ

immersive image can be created. By combining this with technology to determine location and orientation of the

viewer, the basic ingredients of augmented reality are there. Many different layers of information can be added,

resulting in a better and integral oversight of the construction plan (different aspects ant the same time). This was

already the case six years ago (see Piekarski 2003 and Dias 2003).

2. SOLUTION APPROACH

On current day construction sites a augmented reality outfit like the one described above is not common yet. This

may because of the fact that the technology is less wearable in practice. Also, virtual environements and augmented

reality tend to cause simulator sickness, related to motion sickness (see for example Groet et Bos 2008 ). It could

also because of the fact that four years ago computers were not powerful enough with respect to battery power. In

this article however we will not provide the reader with an in-depth analysis on why this is the case. We will provide

a description on the approaches we took at end of 2008 and in the beginning of 2009 using more recent and higher

precision technology. This enabled us to learn about the barriers that keep augmented reality from helping workers

at the construction site and also how we could remove those barriers. By trying to prototype two versions of an

actual working system, we tried to identify all important issues. The contrast between these two versions helped us

to get an even deeper understanding.

2.1 3D-virtual reality on top of a real-life landscape

Just as in earlier approaches we decided to use 3D virtual reality and combine this

with a view of the real-life landscape, captured by a camera image. We decided not

to use an HMD to start with in order to project this augmented reality, because of

several reasons. A tablet sized PC (dubbed the C2B device) can be used without

having to put on an wearable computing outfit, also it can easily be handed over –

especially if in case of a tablet PC - to other people in case of a group experience.

Interaction with such is relatively easy through its tactical interface. Furthermore

tablet PCs are there in many different sizes of scale (e.g. notebook, laptop,

Smartphone, PDA, etc.). And last but not least: the position method in one of the

two approaches required us to add some extra components that would not fit to the

HMD that well. The one drawback is the fact that the hands are occupied. This means that the C2B device is less

useful for construction workers and more useful for inspection workers or architects wanting to share an experience.

In the future, when the very accurate method of determining location and orientation has even more portable

components, we could add them to a HMD also.

2.2 Generic conceptual model

Our generic conceptual process model we use to create an

augmented (or mixed) reality is a continuously looping

workflow containing 6 steps. First we retrieve the location

of the viewpoint in reality and the orientation of the

viewer (a camera) at the viewpoint. Then we retrieve an

image from the viewer. Using the location and orientation

information of the viewer we compute what the

(augmented) view would look like from that position at

that orientation in virtual reality. This view is finally

visually merged and presented to the viewer. Then, we

start at the first step again. In figure 1 there is another step

called ‘retrieve telemetry’. This shows were real live

297


measurement data (e.g. from sensors) could be entered in the virtual reality to create more accurate data.

Due to the enormous development (progression in precision) in localization systems, it is important to be able to

integrate the newest developments available on the market, therefore steps 1,2 are implemented as separated

components that are separated (‘loosely coupled’) from the other components. This gives the flexibility to make an

interface for virtually every localization device, for instance iGPS delivers position and orientation, but differential

GPS for example only position and special sensors are needed for orientation. This interface connects to the main

program with a fixed protocol. Note that the localization and orientation must need to deliver point and vector data.

Information about the location in an area, like the information from an RFID based system like the one used in by

Sanghyung Ahn et al. (Sanghyung Ahn 2008), is not enough for the visual augmented reality as we see them.

Multiple tracking technologies (optical, inertial, GPS) could be used to improve accuracy. The location and

orientation components would then use these multiple technologies and provide a more accurate estimate to the

other components.

This generic conceptual process model was implemented in two ways, to explore different aspects of augmented

reality. Both implementations of the C2B system consisted of a laptop computer, a camera, sensors for determining

location and orientation, and software to integrate the hardware and combine the information as described in the

process model above.

2.3 The semi-fixed high-precision approach

First we took an approach where a high degree of accuracy was quintessential: preferably < 1mm. This kind of

accuracy is required for in situ verification augmented reality applications. The viewpoint view in this case was a

Prosilica-camera placed at the back of a laptop. The image of this camera (1024x768) was not recorded, but

streamed to the laptop display (live). This made the laptop look like a ‘see through’ laptop (as if there was no screen,

but the user was looking right trough a window).

2.3.1 Retrieving viewpoint location and orientation

The high-precision localization and orientation was realized using

a system called iGPS. This system consists of a set of laser

transmitters and at least one double receiver. Both receivers and

transmitters are wirelessly connected to a base station, which

calculates the position of the camera which is at the back of the

laptop. Due to the fact that the system is based on laser there had to

be a Line of Sight (LoS) between receiver and transmitter. And

since we wanted our users to be able to walk around with the C2B,

the body of the user could block the Line of Sight (LoS). Therefore

we added some redundancy to our prototype. It was equipped with

4 transmitters and 4 double receivers (attached to the laptop). In

theory one double receiver and 1,5 transmitters are enough to

calculate the relative position in the covered area with a precision

of less then 1 mm and an angular precision less than 0.1 degree.

The double receiver is about 10 cm height and the two black bands are the actual laser receivers (360ºhorizontal and

120º vertical receiving angle). We wanted total freedom of movement for the user and the vertical angle was

limiting the operational space. Therefore two double receivers were perpendicular mounted on the laptop enabling

all rotational angles. Due to the earlier mentioned Line of Sight problems, this construction was doubled on both

sides (left and right) of the laptop. Because of this elaborate construction the user can walk around, rotate and tilt the

laptop. The wireless setup gave great flexibility in the setting up the environment and freedom of movement for the

user. Finally the base station calculated the position of the camera (viewpoint location) and the angles at which the

laptop was tilted and turned (viewpoint orientation), based on the data retrieved from the sensors.

According to the system vendor Metris this system could cover an area of 300 x 300 meters (indoor & outdoor),

which is relatively large compared to most other tracking systems. Also multiple users are allowed. This means that

several people could work in the same area with a C2B, all with <1 mm accuracy. At the moment the authors of this

article do not know of any other usable system that has a higher accuracy in a large volume (> 10m x 10m). Note

298


that if (exterior) walls are built and sensors the transmitters need to be placed inside these walls and (auto-)

recalibrate the system in order to recreate enough Line of Sights.

2.3.2 Computing and mixing the augmented view

In order to superimpose a view of a virtual reality on top of a real camera feed, we first needed to construct that 3D

virtual reality. This was done use OpenSceneGraph (open source library). All of the components are more or less

standards and the big challenge was integrating these into one program. A big advantage of the OSG-library is the

capability of importing files from many different CAD-programs and if there is no direct import possible, there are

many (free and open) convertors available on the internet. The image overlay we used was a CAD-drawing. Using

the OSG-library implies the usage of real-life coordinate systems and since the iGPS system also uses a real-life

coordinate system, hardly any conversions had to be made. Moving 1 meter in real-life is also 1 meter movement in

OSG and vice versa. The above shows that the realization of the software was relative easy because of the use of

standard components. The update rate of the whole system was at least 30 Hz, giving the user enough interaction to

move freely around.

This virtual world was then finally superimposed on the ‘grabbed’ video image from the camera using the GigE

Vision SDK (a de facto standard in Machine Vision) for video grabbing and presentation. Microsoft Foundation

Classes were used as a basis to combine all components into an application.

2.4 The mobile low-precision approach

For the second approach we assumed there is far less need for precision and more need for mobility and the ability

to get an impression of ‘how a 3D structure would look like in reality’. This system would be more suited for the in

situ experience and warning type of applications. The viewpoint in this case was a Hercules Classic Silver

webcam. This is regular camera that can operate with a resolutions ranging from 320x240 to a maximum of 800x600

(with interpolation). With an USB cable it was connected to a laptop. It had a refresh rate of at least 1 image per

second, depending on the speed of the laptop we were using (Dell Latitude D800 or a Dell Tablet XT).

2.4.1 Retrieving viewpoint location

For this approach we wanted to impose as less restrictions on the out-door

environment as possible. So the use of special two-dimensional barcodes like

those used in the Augmented Reality Toolkit (ARToolkit) was forbidden in this

experiment. Using features of well known environments as used in the

MOBVIS project did require an existing environment, so that was neither

suitable. Since we also assumed that we would do out-door use only with a

clean Line of Sight to enough satellites, so we could use a Global Navigation

Satellite System.

We decided we did want to have sub meter accuracy. This ruled out ‘standard’

GPS, which can be inaccurate for as much as 10 meters. Also, we needed the

ability to interface with a laptop and keep the entire setup as mobile as possible.

This made us select a Trimble® SPS351 DGPS/Beacon Receiver. It makes use

of a technology called differential GPS (dGPS) and uses extra beacons (on

earth) to correct for any possible errors. In our case a mobile phone could be

used for a mobile Internet connection that provided the error correcting signals.

Which is in fact a box with electronical hardware for computational purposes,

with a form factor with dimensions 24 cm length, × 12 cm width × 5 cm depth.

The receiver is accompanied by an antenna (the GA530), which is mounted on

a pole (about 2 meters high) and is connected to the receiver by cables. The entire setup updated its position at least

once per second. It weighed about 2 to 3 kilograms. Using a Bluetooth connection and an Application Programmers

Interface we were able to transmit the position information to our application on the laptop.

Note that this system does not deliver the same accuracy in all directions. In the horizontal range it is less than 1

meter. In the vertical range it is less than 5 meters (according to the supplier Trimble). This is a significant shift for

actual buildings. Therefore some kind of calibration has to be carried out with a basic reference object in reality that

overlaps with an object in virtual reality.

299


In this setup the position of the receiver (i.e. the antenna) is (just like in the high-precision experiment) not the exact

same position of the viewer (the camera). Depending on where the camera is mounted on the total setup, corrections

for the actual location need to be carried out. If the camera is mounted on top of or below the antenna, only a small

correction for height is necessary. The moment however the camera is mounted sideways to the pole, the rotation of

the pole (in the horizontal plane keeping the pole vertically all the time) comes into the equation. If the pole is not in

an upright position calculating the actual position of the viewer also requires the angle that the pole is making with

respect to the ground. This would mean including information on the orientation of the pole.

Note that the GPS signal can be sometimes be blocked at a building site. The use of Dead Reckoning Modules

(DRM) based on inertia-based technology would help provide location information during (temporary) loss of the

GPS signal.

2.4.2 Retrieving viewpoint orientation

In order to feed our computer application with real-time orientation information, we decided to use the ‘OS5000-US

solid state tilt compensated three axis digital compass’. This is a square (1”) piece of electronics with a 0.3” depth

and provides information on the angle of rotation across three axes. There is the compass axis (based on the earth

magnetic field), which provides a fixed reference frame in the horizontal plane. Then there are two axis of tilt (under

a 90 degree angle). This provided us with an orientation vector anywhere on the globe. Note that irregularities in the

earth magnetic field influence this device, as well as other magnetic sources. The device has about a 1 degree

accuracy (in a 360 degree system) – given a maximum 30° tilt, which seemed tolerable for buildings that were in

relatively close range.

At a distance of 1 meter an inaccuracy of 1 degree in a certain direction means a deviation of the focal point of the

viewer of 2 centimeters in that direction. At 100 meters a deviation of 1,75 meters can occur. This means that a

building in virtual reality - which is at 100 meters from the viewer - might seem to float in the air when projected on

top of a live camera feed, assuming that the position of the viewer is measured exactly. When comparing this to the

inaccuracy of the location determination (1 meter in horizontal plane, 5 meters in vertical plane), it immediately

follows that the inaccuracy of the orientation is of the same scale somewhere at a distance of several hundred

meters.

This device was connected to the laptop using a virtual COM (serial across USB) connection. It updated the

orientation information at least once every second. By firmly attaching this square device on top of the camera and

have the tilt-axes correctly align with the lens of the camera, we were able to determine the orientation of the

camera. Note that in order retrieve tilt information about the pole, the camera with the OS5000 device had to be

firmly attached in an upright position to the pole also.

Note that deviations in the surrounding magnetic field (e.g. influence of large steel bars) could result in further

errors since the digital compass is based on magnetic sensors. We did not examine this, but suspect this effect is

noticeable in the presence of structures that impact the earth magnetic field. We also suspect that a (digital)

gyroscopic compass / tilt sensor would probably deliver better results. When such a device would become available

to use in the same form-factor as the OS5000, we would be interested in comparing them. .

2.4.3 Computing and mixing the augmented view

Construction construct of the virtual reality was done using the building blocks of Microsoft Windows Presentation

Foundation (WPF) on the Microsoft .NET platform version 3.5. It provides developers with a unified programming

model for building so called ‘rich Windows smart client user experiences that incorporate UI, media, and

documents’. A discussion of this programming framework is beyond the scope of this article. We only deal with 3D

Graphics, a part of the Graphics and Multimedia section of WPF. The framework offers a component (also known as

an element) called the ‘Viewport3D’. It functions as a window - a viewport - into a three-dimensional scene. A 3-D

scene can be projected onto the 2D surface of this element. The way it is projected is determined by the camera

settings of the Viewport3D.

Within a Viewport3D element a programmer can define a camera object. We used the ‘Perspective Camera’ with

vanishing points as in reality. In order to have a proper alignment between the view in the real camera feed and the

virtual world camera, it was important to have the same Field of View (FoV). This can be defined as the ratio

between the amount of horizon that can be seen at a certain distance. In terms of a person standing in front of a

window: if the persons stands close to the window he can see a lot of the outside scenery. If he stands at a greater

300


distance, he can see far less of the scenery outside. Depending on what camera (and of course the lens) one uses, the

FoV changes. Using a calibration board we measured out the distances and determined the FoV of our camera. By

setting set the FieldOfView attribute of the camera object in our application, the FoVs of the real (Hercules) and

virtual (WPF 3D Perspective) camera were aligned.

We then tried to align the origin of the virtual world with a coordinate in the real world. Since we opted for a mobile

approach we had to take into account the effects of being in ‘open wide space’. From a theoretical point of view this

immediately caused problems, because of the essential differences of the coordinate systems. The GPS system uses

polar coordinates, assuming a sphere system. The WPF 3D coordinate system assumes a cubical grid. One can

devise mappings, but they always will result in some kind of distortion of reality. Cartographers have know this for

years – at least in the 2D case - and have come up with many attempts to remove distortion for certain purposes.

Discussion of this is beyond the scope of this article. Although we do not often perceive curvature of the earth, it is

not something you can neglect that easy. For example if you have a perfectly straight bar of 100 meters and the

ground is completely ‘flat’, i.e. it follows the perfect shape of a sphere, the bar will not fit to the ground. Instead, if

you balance it in the middle, both ends will stick out 5 cm above the ground. Because of the inaccuracy of the

location determination and the orientation, we decided this inaccuracy was tolerable for this approach. However,

with objects far away at the distant horizon, the inaccuracy will be noticeably much more severe.

FIG. 1: Camera and coordinate system in WPF 3D graphics (from the Microsoft MSDN website)

In practice we aligned the location of the virtual and real world by choosing an anchor point in reality. That point

was defined as ‘point Zero’ (0,0,0) in the WPF3D coordinate system. We then built a scene in the WPF3D space by

using the 3D drawing primitives (lines, rectangles, etc.). Coordinates in reality were ‘corrected’ by using a delta

between the real world coordinates and the anchor point. Note that the height of the camera was also corrected by a

default distance since it was fixed on a pole. Aligning orientation was done in a similar way: we assumed that the

real (magnetic) south to north direction always was parallel to the z-axis running from the positive to the negative

end. Based on that, a simple scenery of objects was drawn, were it was assumed that the cubical geometry of the

virtual world also applied in reality. Which of course was not true, but acceptable with in the hundred meter range.

For mapping of measurement units in the physical reality to measurement units in virtual reality we took a short cut

and did some manual calibration by drawing several square squares in virtual reality and align this with square

objects in reality.

Finally, using the Extensible Application Markup Language (XAML) within WPF we were able to tell (‘declare’)

the WPF framework to automatically map Viewport3D element onto another element that showed the camera feed

from the Hercules camera. Only the objects from the scenery were visible, there was no background of the virtual

reality with in the Viewport3D, only transparent pixels, which allowed the background reality camera feed to be

seen. The combination of the two resulted in augmented reality.

3. USAGE EXPERIENCE

During the construction of the two prototypes we carried out several tests and could experience the usage of the

developing C2B prototypes first hand.

301


3.1 The semi-fixed high-precision approach

We had the following positive user experiences with respect to the iGPS based system:

• the user can work with a highly accurate augmented reality in a relative large volume. One of the

applications could be checking the placement of the poles used for brick-laying or in an even

advanced way placement of these poles. Another useful application could be laying tiles in complex

(color) patterns; with the prototype the original drawings show the correct tile in real life;

• in our opinion setting up and calibrating the iGPS system was simple and fast compared to many other

large volume tracking devices. It is possible to carry the setup to a construction site, set it up, perform

some calibration and start using the system; and

• the usage of the entire system was intuitive and self explanatory. People tend to ‘get the hang of it’

rather soon, once they perceive the C2B as a window on an augmented reality.

There were also less positive experiences in usage:

• one of the first problems we experienced was Lag-Time: the system did not respond fast enough to

motion. This was introduced by the iGPS system, which had an update rate of 60 Hz, but did not

deliver new location information every 1/60th of a second. It collected a few samples and sent them in

a burst mode. Using new firmware we this problem was solved;

• the need for a Line of Sight restricted the movement of users in the beginning. After repositioning the

receivers, this was nearly eliminated;

• the system is a bit more bulky, compared to the original artists impressions. The iGPS system is the

main reason, the receivers are not very small and they also need boxes with batteries and wireless

transmitters;

• the position of the users head has to be aligned with the axis of the camera, otherwise the real-life

world behind the laptop and the image presented by the camera does not really match and the ‘window

on the world’ effect disappears. A HMD does not suffer from this problem; and

• due to the fact that a Line of Sight is needed, it is less suitable when walls are erected between the

beacons and the receivers (i.e. so much walls that all redundancy is lost).

3.2 The mobile low-precision approach

We had the following positive user experiences with respect to the Differential GPS and tilt-sensor based system:

• while staying in a fixed position with the setup a user can get some kind of impression of an virtual

object in its real surroundings;

• users do not have to set up beacons in advance, since the GNSS satellites have already been put into

place by other people; and

• a user cannot block the line-of-sight between the Differential GPS receiver and himself.

There were also less positive experiences in usage:

• a collection of huge buildings or other large objects will probably block the GPS signal;

• the tilt-sensor can be disturbed by strong magnetic fields;

• the entire setup is mobile, but not light. The pole with receiver, antenna and a laptop can be carried by

one person (as long as everything is taped together securely), but it requires some strength. It also

turned out that the pole is very useful a support for the laptop;

• when moving around the augmented view becomes pretty unstable because of all the components

moving and vibrating. Only when the entire setup is put to the ground again and allowed to ‘rest’ for a

while, a steady image appears again. This seems partially because the tilt-sensor is very sensitive.

302


Small movements create different readings which alter the view. On a relatively slow laptop this is

clearly visible; and

• a 3D virtual object that is behind a real object on the camera will still be projected in front of the

camera image. This pretty much ruins the illusion. The way to solve this is by creating a

representation of the real object in virtual reality and have this object rendered with the texture of the

underlying camera image. We did not carry this out due to time limitations within the project.

4. TOWARDS SELECTION CRITERIA

We state that it would be useful to have selection criteria that predict what approach would be best, depending on

one of the three major categories of application. Based on our experience we think it is too early to come up with

precise selection criteria. We do state several remarks that should bring us towards the selection criteria during

further research. With respect to in situ verification we strongly suspect that the choice for an approach heavily

depends on the level of accuracy a specific building inspector needs. The 10cm error in GPS-position combined with

the 1 degree error in tilt in the mobile low-precision approach can cause deviations of more than 20 centimeters at a

distance of 10 meters. Depending on whether or not this is precise enough, how mobile the inspector wants to be and

how much financial resources are dedicated to inspections, one of the two approaches is the better one. Note that we

did not carry out studies with a HMD. This might cause simulator sickness (see Dodson et al. 2002) and probably is

a criterion of its own.

In situ experience seems to be less demanding than in situ verification, but we think more research should be done

on user experience before selection criteria can be defined. In general we currently cannot provide (precise) numbers

with respect to percentages in deviation between aspects of augmented reality and physical reality. When we showed

the C2B approaches people tended to react differently. While several people reported it as helpful (e.g. ‘it supports

the imagination’), other people focused on the difference in augmented and physical reality (e.g. ‘it is not real

enough’). Because we did not carry out a scientific study on the experience on a representative population, we can

not further elaborate on this.

Finally for in situ warning, we suspect that mobility can be more important than precision when it comes to warning

people in the building area. Also, the chance of inducing motion sickness is important in this approach, since people

would walk around continuously. Both approaches in this article lean on visual feedback. We think that audio or

tactile feedback might be of better use. Think of a sound or a vibration in the presence of fall-through openings for

example.

5. CONCLUDING REMARKS

Our goal was to learn about the barriers that kept augmented reality from helping workers at the construction site

and how we could remove those barriers. By trying to prototype two versions of the C2B system, we have been able

to identify the important issues. Apart from the obvious need for real accuracy in situ verification and also in situ

impression applications, the main challenges of creating C2B seem to be in:

1. Integrating and calibrating all the different components. Especially in the case of the mobile approach the

user has to walk around with all kinds of components connected by cables. We suspect this acts as a barrier

too. The arrival of consumer smartphones with a GPS and a compass build-in seems very interesting in that

respect. Although the accuracy is not at the dGPS level, the usage experience in some areas of application

seems to be good enough. See for example the Layar application (www.layar.eu).

2. Aligning the Virtual World with the Real World. We suggest creating (software) components that take into

account that the world is not flat. For example: towers at larger distances should be less visible due to a

curvature in the earth. From a certain distance the viewer should not be able to see the base of a very high

tower. In further research we would like to investigate the amount of computing power (and energy) needed

for the correction of complicated scenes.

3. Ease of deployment when high precision is demanded. The semi-fixed system still needs setup-time. Once

the walls go up users need to (auto-)recalibrate the site. A solution might be to use a signaling medium that

is less obstructed by physical objects, like radio. To our current knowledge however higher-precision (<10

303


cm range) radio location information systems (e.g. those from Ubisense) still require calibration and initial

setup time. Also, we have no experience how (thick) walls influence the accuracy of these systems. We

would welcome experimental data in those areas.

In order to solve the issue of more accuracy in orientation (in the mobile approach), we need a component that is far

less depending on the earth-magnetic field. This component should fit on a wearable camera and should be low on

the usage of power. We are looking forward to collaboration with organizations which are trying to develop

(hardware) modules like these.

Once we have met these challenges in the future, the C2B system can be made even more powerful by including 4D

renderings (i.e. a 3D model and a schedule). Also we could integrate real-time sensor information to increase the so

called situational awareness of construction workers with recent information.

6. REFERENCES

Piekarski, W. and Thomas, B.H. (2003). “Interactive Augmented Reality Techniques for Construction at a Distance

of 3D Geometry”,

Immersive Project Technology / Eurographics Virtual Environments Conference, Zurich, Switzerland, 22-23

May, http://www.tinmith.net/papers/piekarski-ipt-egve-2003.pdf

Dias, J.M.S., Capo, A.J., Carreras, J. and Galli, R. (2008), A4D: “Augmented Reality 4D System for Architecture

and Building Construction

CONVR 2003: 3rd Conference of Construction Applications of Virtual Reality, Virginia Tech, September

24th – 26th , http://dmi.uib.es/~ajaume/recerca/CONVR2003.pdf.

Sanghyung Ahn, Johnson, L.M., Do Hyoung Shin, Dunston, P.S., and Martinez, J. (2008) “Prevention of

construction accidents with augmented reality”,

CONVR 2008: 8th Conference of Construction Applications of Virtual Reality, Kuala Lumpur, Malaysia,

October 20-21

http://www.convr2008.com/index_files/submissions/Training%20&%20Game-1.pdf

Groen, E., Bos, J.E., (2008), “ Simulator sickness depends on frequency of the simulator motion mismatch: An

observation”, Presence: Teleoperators and Virtual Environments, Volume 17 , Issue 6, Pages 584-593 , MIT

Press Cambridge, MA, USA

Dodson, A., Evans, A., Denby, B., Roberts, G.W., Hollands, R., Cooper, S., (2002), “Look Beneath the Surface

with Augmented Reality”, GPS World,

http://www.gpsworld.com/gps/application-challenge/look-beneath-surface-with-augmented-reality-726

Trimble® SPS351 DGPS/Beacon Receiver, http://www.trimble.com/sps351.shtml

Ocean Server 5000 Family http://www.ocean-server.com/download/Compass_OS5000_Family.pdf

Hercules Classic Silver webcam http://www.hercules.com/uk/webcam/bdd/p/22/hercules-classic-silver/

Windows Presentation Foundation, http://windowsclient.net/wpf/white-papers/when-to-adopt-wpf.aspx

MOBVIS, http://cordis.europa.eu/ictresults/index.cfm/section/news/tpl/article/id/90340

iGPS, http://www.metris.com/large_volume_tracking__positioning/basics_of_igps/

ARToolkit, http://www.hitl.washington.edu/artoolkit/

Ubisense, http://www.ubisense.net/pdf/fact-sheets/products/software/Ubisense-Precise-Location-062409-EN.pdf

304


DEVELOPMENT OF A ROAD TRAFFIC NOISE ESTIMATION SYSTEM USING VIRTUAL REALITY TECHNOLOGY

Shinji Tajika, Graduate Student,

Department of Civil and Environmental Engineering, Chuo University

[email protected]

Kazuo Kashiyama, Professor,


[email protected]

Masayuki Shimura, General Engineer,

Civil Engineering and Eco-Technology Consultants

[email protected]

ABSTRACT: This paper presents a road traffic noise estimation system using virtual reality (VR) technology. This

system exposes an observer in VR space to the acoustic information of the road traffic noise synchronized with

vehicle CG animation. Road traffic noise is computed based on the geometric acoustics theory. The IPT (Immersive

Projection Technology) is employed in order to create VR space, and CG animation is also created by OpenGL. The

observer can change the road environment and the vehicle condition in VR space by the operation of the controller.

The acoustic information created by the present system is calibrated to correspond with the acoustic information

measured by the sound level meter. The present system is shown to be a useful tool to predict the road traffic noise

in planning and designing stage of road.

KEYWORDS: Road traffic noise, audio-visual stimuli, geometric acoustic theory, immersive projection technology

1. INTRODUCTION

It is important to estimate the noise level for the planning and designing of the road and the sound barrier, because

the road traffic noise may cause stress, disordered sleep and defective hearing to the inhabitants who live adjacent to

the roads. Recently, in order to estimate the road traffic noise, a numerical simulation based on the geometric

acoustic or wave acoustic theories has been developed. Generally, the numerical results are visualized using

computer graphics (CG). However, it is difficult to understand the noise level intuitively, because the visualization is

not a suitable tool to express the acoustic information. The tool that can present the computational result for acoustic

information is necessary for the citizens who are not the acoustic specialists. So we are developing the virtual

experience system about road traffic noise.

In the past studies, several systems that expose road traffic noise as the acoustic information have been presented.

Nagano et al. (1999) developed a system that allows the user to hear noise by manipulating the regeneration levels

of recorded video and sound. Makanae et al. (2004) developed a simulator that allows the user to hear noise in

synchronization with traffic flow simulation. Mourant et al. (2003) developed several traffic noise simulators for

driving simulator. However, there are quite few attempts to develop a system that allows the user to hear noise in

various conditions and to view a vehicle animation of stereoscopic view and real scale like reality.

This paper presents a road traffic noise estimation system using virtual reality (VR) technology. This system

presents the acoustic information of the road traffic noise based on computational results and the visual information

of the vehicle CG animation to observer in VR space. In order to compute the road traffic noise level, the ASJ RTN-

Model 2008 (Research Committee of Road Traffic Noise in the Acoustical Society of Japan : 2009) based on the

geometric acoustic theory is employed. The VR space is created by the IPT (Immersive Projection Technology)

(Wegman et al. : 2002) and the active stereo method is employed for stereoscopic view. The display function of

interface is developed to realize that the observer can change the road environment and the vehicle conditions in VR

space. In order to check the agreement between acoustic information created by the present system and the

computational results, the acoustic information is measured by a sound level meter. The present system is shown to

be a useful tool to predict the road traffic noise in planning and designing stage of road.

305


2. A ROAD TRAFFIC NOISE ESTIMATION SYSTEM USING VR TECHNOLOGY

Figure 1 shows the concept of the present system. The system provides the vehicle driving animation by the real

scale stereoscopic view in VR space which is realized using the IPT. And, the acoustic information of road traffic

noise based on numerical result which is synchronized with the movement of vehicle is produced. The present

system has following three characteristics. First, the observer can move to arbitrary position and can hear the road

traffic noise that correspond with the position, since the road traffic noise level is computed using the position of

observer in real-time (see (A) in Figure 1). Second, the observer can change the road environment; height of sound

barrier, pavement type and passage years after pavement ((B) in Figure 1). Third, the observer can change the

vehicle conditions; vehicle type, vehicle speed and running distance of vehicle ((C) in Figure 1). Furthermore, the

display function of interface is developed in order to realize the second and third characteristics. The procedure of

creation of VR space is shown in Figure 2.

2.1 Making of vehicle animation

The CG data of the vehicle and sound barrier is created by 3DCG software (3dsMax : Autodesk). The polygons of

CG data are reduced in order to speed up the rendering CG time. The CG data is converted to OpenGL format and

the vehicle driving animation is created by OpenGL. The start of vehicle driving animation is synchronized with the

operation of controller.

VR space

noise

observer

Observer can hear

the road traffic noise.

VR space

Observer can change

the road environment.

VR space

Observer can move to

arbitrary position.

VR space

Observer can change

the vehicle condition.

Speed up

movement Sound barrier

(A) (B) (C)

VR space

noise

observer

Observer can hear

the road traffic noise.

VR space

Observer can change

the road environment.

VR space

Observer can move to

arbitrary position.

VR space

Observer can change

the vehicle condition.

Speed up

movement Sound barrier

(A) (B) (C)

FIG. 1: Concept of present system

Road environment data

Vehicle condition data

Observer ’s position data

Vehicle ’s position data

Computation of noise

Making of vehicle

animation

Presentation of road

traffic noise by hearing

information

Creation of VR space

Road environment data

Vehicle condition data

Observer ’s position data

Vehicle ’s position data

Computation of noise

Making of vehicle

animation

Presentation of road

traffic noise by hearing

information

Creation of VR space

FIG. 2: Procedure of creation of VR space

VR space

306


2.2 Create road traffic noise using the acoustic information

In the present system, the road traffic noise level is computed by the ASJ-RTN Model 2008. The change of

frequency by the Doppler effect is also considered. The computation is performed using the position data of the

vehicle and the observer in real-time, and the acoustic information based on numerical results is created by using

MAX/MSP (Cycling '74 ) which is a programming software for music and multi-media.

2.2.1 ASJ RTN-Model 2008

The ASJ RTN-Model 2008 is a model for road traffic noise based on the geometric acoustic theory, which is

developed by the Acoustic Society of Japan. The concept of ASJ RTN-Model 2008 is shown in figure 3. In the

model, the A-weighted sound power level of road vehicle noise is computed and then the sound propagation is

computed. In the sound propagation, two kinds of sounds, a direct sound and a reflected sound, are computed. The

road traffic noise at observer’s position is evaluated by synthesizing of direct sound and reflection sound.

① Computation of A-weighted sound power level of road vehicle noise

The A-weighted sound power level of road vehicle noise WAL is evaluated as.

where a is the constant related to the vehicle type, b is the coefficient related to the vehicle speed, V is the vehicle

speed, C is the correction term which is expressed as follows.

where surfLΔ , gradLΔ , dirLΔ ,

etcLΔ are the corrections concerning with the noise reduction with drainage

pavement etc, the change of road vehicle noise by the vertical slope, the directivity of vehicle noise, the rest factors,

respectively. In this paper, the express way is assumed and surfLΔ of each vehicle type is given as.

�more than 60km/h (vehicle speed):

small vehicle (standard + subcompact size ):

� large vehicle (medium + large size ):

�less than 60km/h (vehicle speed):

small vehicle (standard + subcompact size):

CVbaLWA

++= 10log �1�

)1(log4.6log52.3 1010 ++−=Δ yVLsurf �3�

)1(log6.3log50.5 1010 ++−=Δ yVLsurf �4�

etcdirgradsurf LLLLC Δ+Δ+Δ+Δ= �2�

)1(log4.67.5 10 ++−=Δ yLsurf �5�

�Computation of A-weighted sound power level

of road vehicle noise

Observer

(sound receiving point)

Sound barrierSound barrier

Vehicle

(sound source)�Computation of

sound propagation

(direct sound)

�Computation of

sound propagation

(reflection sound)

�synthesis of direct sound

and reflection sound

(Computation of A-weighted sound pressure level

at sound receiving points)

�Computation of A-weighted sound power level

of road vehicle noise

Observer



Vehicle

(sound source)�Computation of

sound propagation

(direct sound)

�Computation of

sound propagation

(reflection sound)

�synthesis of direct sound

and reflection sound

(Computation of A-weighted sound pressure level

at sound receiving points)

FIG. 3: ASJ RTN-Model2008

307


large vehicle (medium + large size):

where y is the passage years of pavement, V is the vehicle speed.

� The calculation of sound propagation (direct sound)

The A-weighted sound pressure level d

AL of direct sound which is propagated from vehicle is evaluated as:

where WAL is the A-weighted sound power level of road vehicle noise, r is the distance in a straight line

between observer and vehicle, corLΔ is the correction concerning with attenuation factors that influences sound

propagation.

Where corLΔ is expressed as follows.

in which difLΔ , grndLΔ , airLΔ are the corrections concerning with the attenuation caused by diffraction, the

attenuation caused by grand effect, the attenuation caused by atmospheric absorption, respectively.

In this paper, the length of the sound barrier is assumed to be infinity and difLΔ is given as follows.

where specC is the constant related to the pavement type, δ is a different length between a straight path and a

diffraction path from the observer to the vehicle ( see figure 4 ). In the equation (9), if the sound source can be seen

from the observer directly, the sign ofδ is to be a minus, and if a is larger thanb , ],min[ ba is to be a .

� The calculation of sound propagation (reflection sound)

The A-weighted sound pressure level r

AL of reflection sound is computed by using mirror image of sound source.

The position of mirror image sound source is assumed to be a position of symmetry of real sound source as shown in

Diffraction path

Straight line path

Vehicle

(sound source)

Observer


Sound barrier

1≥δspecC�

10 <≤ δspecC0<δspecC

Diffraction path

Straight line path

Vehicle

(sound source)

Observer



1≥δspecC�

10 <≤ δspecC0<δspecC

FIG.4: Diffraction of sound

corWA

d

ALrLL Δ+−−= 10log208 �7�

airgrnddifcor LLLL Δ+Δ+Δ=Δ �8�

0

10

1

])(sinh0.175,0min[

)(sinh0.175

)(log1020

414.01

414.01

10

<

<≤

≥

+−

−−

−−

=Δ−

−

δ

δ

δ

δ

δ

δ

spec

spec

spec

spec

spec

spec

dif

C

C

C

C

C

C

L � � � �9�

�6� )1(log6.39.3 10 ++−=Δ yLsurf

308


figure 5. The A-weighted sound power level of mirror image sound source is computed by equations (1)�(6), and

the sound propagation is computed by equations (7)�(9).

� Synthesis of direct sound and reflection sound

The A-weighted sound pressure level AL at observer’s position is evaluated as follows.

where d

AL and

r

AL are the sound pressure level of real sound source and mirror image sound source, respectively.

2.2.2 The Doppler effect

The Change of frequency by the Doppler effect is considered as follows.

here f is frequency of sound source, U is sonic speed, su is velocity of sound source (vehicle), θ is angle

between straight line from sound source to observer and direction of movement of sound source (see figure 6).

2.2.3 Presentation of road traffic noise by hearing information

The MAX/MSP (Akamatsu et al. : 2006) is employed in order to create the acoustic information of road traffic noise

computed by the ASJ RTN-Model 2008. Three kinds of input data are prepared for MAX/MSP, the road traffic

noise level, frequency and the wave file of vehicle noise. These data is captured by MAX/MSP all the time, for this

reason, the acoustic information can reflect the computational noise level and frequency in real-time.

2.3 Virtual Reality System

2.3.1 Outline of VR system

The IPT (Immersive Projection Technology) is employed for VR technology and the immersive display is employed

for VR display. Figure 7 (a) shows the VR system “HoloStage” in Chuo University and Figure 7 (b) shows the VR

Real sound

source

ObserverObserverMirror image

sound source


Reflection sound

Real sound

source

x x

Real sound

source


sound source


Reflection sound

Real sound

sourceReal sound

source


sound source


Reflection sound

Real sound

source

x xFIG.5: Mirror image sound source

+= 1010

10 1010log10

r

A

d

A LL

AL �10�

−=′

θcossuU

Uff �11�

FIG.6: Computation of the Doppler effect

su

θcossu

θ

Sound source

observer

su

θcossu

θ

Sound source

observer

309


projector and display system. This system is composed of three large and flat screens and high-performance

projectors corresponding to the screen. The front and side screens are transmissive ones and the bottom screen is

reflective one. The VR space is created by projecting the image on the front and side screens, and the bottom screen

as shown in Figure 7 (b). This system has 7.1ch sound speakers and the VR space is created by the acoustic

information and the visual information.

2.3.2 Computer hardware and network

The HoloStage has a PC cluster system consists of one master-PC and four slave-PCs. The specifications of the PC

cluster are shown in Table 1. The Giga-bit Ethernet is employed for the network of PC cluster. Figure 8 shows the

network configurations. A slave-PC computes the coordinates of location of viewpoint sequentially. Other 3 slave-

PCs create the stereoscopic image from the view point rapidly.

2.3.3 Tracking system

The observer’s motion is captured by a system called VICON Tracking system, which is the optics type motion

tracking system. The positions of markers fitted to liquid crystal shutter glasses and controller are tracked by the

��*+ )��

��!"& ��/.��')#*+�+"'&

�� ,�$��') � ��(+ )'&��+%��1

� %')0 ��3

�)�(!"�*��)� �-"�"��,��)'��2�

�$�- ��2

��!"& ��/.��')#*+�+"'&

�� ,�$��') � ��(+ )'&��+%��1

� %')0 ��3

�)�(!"�*��)� �-"�"��,��)'��2�

TAB.1: Specifications of PC Cluster System

FIG.8: Network Configuration

MasterPC

HUB

SlavePC

SlavePC1

SlavePC2

SlavePC3

Projector1

Projector2

Projector3

Amp. 7.1chSpeaker

Front

screen

VR display

Bottom screen

Side

screen

MasterPC

HUB

SlavePC

SlavePC1

SlavePC2

SlavePC3

Projector1

Projector2

Projector3

Amp. 7.1chSpeaker

Front

screen

VR display

Bottom screen

Side

screen

FIG.7: VR system based on IPT

(a)

mirror

front screen

bottom screen

side screen

projector VR space

(b)

310


tracker device. Figure 9 (a) shows a VICON tracker device. The 6 VICONs are provided in the 3D VR space

surrounded by the three screens. Figure 9 (b) and (c) show the liquid crystal shutter glasses and controller used by

observer. In these figures, the white small ball denotes the marker.

2.3.4 Method of stereoscopic view

The binocular parallax is employed for the stereoscopic view. The stereoscopic view is realized in VR space by

creating the image that corresponds to binocular retinal images, and projecting it to screen. In this system, the active

stereo method employed to realize the stereoscopic view. The observer wear the shutter glasses shown in Figure 9

(b), which are synchronized to the computer display through infrared emitters alternating the left and right eye

viewpoints at 120 Hz. The observer’s brain combines the two views into a 3D stereoscopic image.

2.3.5 7.1ch sound speakers

The Holostage has a 7.1ch sound speaker system (7 speakers and a sub-woofer). Figure 10 shows the layout of the

system. The speakers are connected to the master-PC (see Figure 8) and output the sound which is reproduce by

master-PC. Each speaker outputs the same sound in the present system.

2.4 Interface function

In order to change the road environment and the vehicle condition in the VR space easily, the interface function was

developed using OpenGL. The interface is displayed in the VR space (see Figure 11 (a) and (b)) by operating the

controller which is shown in Figure 9 (c). The interface position is fixed on the front screen as the observer can

watch the interface easily. The observer can change road environment and vehicle conditions as; the height of the

sound barrier (0�5m), the pavement type (drainage pavement or dense-graded asphalt concrete pavement) and the

passage years after pavement (0�20years), vehicle type (standard size or semi-compact size or medium size or

large size), vehicle speed (50�100km/h). And, these changes for road environment and vehicle conditions are

reflected in the acoustic and visual information in real-time. Also, other interface function was developed in order to

confirm the selected conditions and the A-weighted sound pressure level at observer’s position (Figure 11 (c)).

��

��

��

��

��

FIG.10: layout of 7.1ch sound speakers

FIG.9: VICONtracker(a), liquid crystal shutter glasses(b) and controller(c)

(a) (b) (c)

311


3. CREATION OF VR SPACE

3.1 Calibration of acoustic information

In order to verify whether the computational sound pressure level agree with the traffic noise from the stereo

speaker, the output sound was measured by the sound level meter (LA-2560 : ONOSOKKI ). The road environment

and the vehicle conditions selected are shown in Table 2. Figure 12 shows the seen of measuring sound level. The

sound level meter was set up on height of 1.4m. Figure 13 (a) shows the unit pattern of the computational and

measuring results after calibration. The input data of noise level to MAX/MSP was calibrated to match the

computational results to measuring results. From this figure, it can be seen that the computational results are in good

agreement with measuring results.

FIG.11: Interface function

(c) Display of conditions and noise level

(a)Selection of changing items (b) Selection of condition

FIG.12: Measurement of A-weighted sound power level

TAB.2: Road environment and vehicle condition

Height of sound barrier : 0m

Vehicle type : standard car

Vehicle speed : 50km/h

Pavement type : drainage pavement

Passage years after pavement : 5 years

Horizontal distance from traffic

lane to observation point: 5m

Height of sound barrier : 0m

Vehicle type : standard car

Vehicle speed : 50km/h

Pavement type : drainage pavement

Passage years after pavement : 5 years

Horizontal distance from traffic

lane to observation point: 5m

312


3.2 Creation of VR space

Figure 14 shows the situation of hearing the road traffic noise based on computational result in VR space. The

observer can hear the road traffic noise considering the Doppler effect, which synchronized with the movement of

the vehicle from arbitrary position by wearing the liquid crystal shutter glasses and operating the controller. Also,

the observer can share the acoustic information and visual information in VR space with other observers at same

time. Besides, the observer can change the road environment and the vehicle conditions, and can hear the road traffic

noise in accordance with the change. Figure 14 (b) shows the scene changing the height of sound barrier. The

observer can hear the road traffic noise that is decreased by sound barrier. Figure 13 (b) shows the unit pattern of the

computational and measuring results with sound barrier (the height of sound barrier and vehicle speed is assumed to

be 3m and 100km/hr respectively). From the comparative studies in Figure 13, it can be seen that the computed

results are good agreement with the measuring results. From the results, it can be seen that the observer can

understand the noise level accurately by the present system, because the noise level is presented by the acoustic

information.

4. CONCLUSIONS

A road traffic noise estimation system using VR technology has been presented in this paper. The road traffic noise

has been computed based on the geometric acoustics theory and presented by acoustic information in VR space. The

acoustic information has been measured by the noise level meter, and calibrated to correspond with the numerical

result of road traffic noise level. The interface function has been developed in order to change the road environment

and the vehicle conditions in VR space. The key features of the present system are as follows.

FIG.14: A situation of hearing the road traffic noise

(a) (b)

FIG.13: Unit Pattern

(a) (b)

�

��

�

�

��

� � �

��

��

��

��

��

��

��

�

��

�

�

��

� � � �

��

��

��

��

��

��

��

313


�As the present system provides the visual and the acoustic information in VR space, the observer can understand

the noise level easily by hearing the noise, and also can share the information with other observers at same time.

�The real time simulation is realized by using the controller which can change the vehicle condition and the road

environment in VR space.

From the results obtained in this paper, it can be concluded that the present system provides a useful tool to predict

the road traffic noise in planning and designing stage of road. The verification of the applicability of the present

system to the more complicated road environment is left in the future work.

5. REFERENCES

Nagano T., Yasuda Y., Furihata K. and Yanagisawa T. (1999). "The examination of virtual reality system for noise

experience", The Technical Report of The Proceeding of The IEIC, EA99-56, pp.33-40.

Makane K. and Takahashi K. (2004). " 3D Traffic Noise Simulator for VR Environment." CONVR 2004: 4th

Conference of Construction Applications of Virtual Reality, Lisbon, Portugal, Sept. 14-15, 183-186.

Mourant R.R. and Refsland D. (2003). "Developing a 3D Sound Environment For a Driving Simulator",

Proceedings of the Ninth International Conference on Virtual Systems and Multimedia, Montreal, Canada,

October, pp. 711-719.

Research Committee of Road Traffic Noise in the Acoustical Society of Japan. (2009). " Road traffic noise

prediction model “ASJ RTN-Model 2008” proposed by the Acoustical Society of Japan", Jounal of Japan

Acoustical Society of Japan, Volume 65, pp. 179-232.

Wegman E.J. and Symanzik J. (2002). " Immersive Projection Technology for Visual Data Mining.", Jounal of

Computational & Graphical Statistics, Volume 11, Number1, March 1, pp. 163-188.

Akamatsu M. and Sakonda S.(2006). “2061 : A Max Odyssey”, Rittor Music.

314


APPLICATION OF VR TECHNIQUE TO PRE- AND POST-PROCESSING FOR WIND FLOW SIMULATION IN URBAN AREA

Kazuo Kashiyama, Professor,


[email protected]

Tomosato Takada, Graduate student,


[email protected]

Tasuku Yamazaki, Graduate student,

Department of Civil and Environmental Engineering, Chou University

[email protected]

Akira Kageyama, Professor,

Department of Computer Science and Systems Engineering, Kobe University

[email protected]

Nobuaki Ohno, Scientist,

Earth Simulator Center, Japan Agency for Marine-Earth Science and Technology

[email protected]

Hideo Miyachi, General Manager,

Visualization Division, KGT Inc.

[email protected]

ABSTRACT: This paper investigates the application of the virtual reality technique to pre- and post processing for

wind flow in urban area. The accurate shape model for landform and buildings is prepared by the integration of

several GIS and CAD data and an automatic mesh generation method is developed. The visualization method based

on VR technique is employed for both pre- and post-processing to understand the quality of mesh and flow field. The

present system is shown to be a useful tool to investigate the wind flow in urban area.

KEYWORDS: Flow visualization, virtual reality, wind flow, GIS/CAD, VFIVE

1. INTRODUCTION

Numerical simulation is becoming more popular to evaluate the wind flow in urban area in accordance with the

development of hard- and software of computers. However, the following problems are pointed out in practical

computations of this type of problems as: 1) it is difficult to check the quality of shape model and mesh for the

complicated spatial domain; 2) it is difficult to understand the three dimensional structure of flow field since the

computational results normally express on the screen or display.

In order to overcome those problems, this paper investigates the application of virtual reality technique to pre- and

post processing for wind flow in urban area. The accurate shape model for landform and buildings is prepared by the

integration of several GIS and CAD data and an automatic mesh generation method based on Delaunay method is

developed. The visualization method based on VR technique is employed for pre and post processing to understand

the three dimensional structure of mesh and flow field. The attempt to combine the different CG images created by

different visualization software is also performed.

The present system is applied to the simulation of wind flow in urban area in Tokyo and is shown to a be useful tool

to investigate the wind flow in urban area at the stage of planning and designing.

315


2. VR ENVIRONMENTS

The visualization based on virtual reality (VR) is employed to�pre- and post-processing for wind flow simulation in

urban area. Fig. 1 shows the exterior view of the VR system “HoloStage” based on IPT (Immersive Projection

Technology). This system consists of a PC cluster (1 master-PC and 4 slave-PC), 3 projectors, VR display (3 large

screens) and a position (head) tracking system (see Fig. 2). The stereoscopic image from the arbitrary viewpoint of

observer is displayed in VR space (CAVE room) by the position tracking system.�The details of the VR system are

described in the reference (Takada and Kashiyama, 2008).

The binocular parallax is employed for the stereoscopic view. The stereoscopic view is realized in VR space by

creating the image that corresponds to binocular retinal images, and projecting it to screen. The active stereo method

employed for the method of stereoscopic view.

FIG.1: VR system “HoloStage”

FIG.2: Hardware configuration

3. PRE-PROCESSING USING VR TEQUNIQUE

In order to obtain the accurate numerical solution, it is important to create an accurate shape model and a mesh with

good quality. The modeling method using GIS/CAD data is developed. The present method is applied to the

Nihonbashi-area, Tokyo.

In case of the modeling of urban area, the building data is needed in addition with the elevation data. For the data for

buildings, the GIS data (Mapple 2500) obtained by the aerial-photo and -laser surveying are employed. For the data

of land elevation, the digital elevation map issued by the Japanese geographical survey institute is employed. Fig.3

shows the configuration of numerical wind tunnel. The studied urban area is modeled on the circle area. The

arbitrary wind direction is treated by the rotation of the circle area. Fig.4 shows the 3D shape model obtained by

GIS/CAD data. The vertical shape for low-storied buildings is assumed to be straight and that for the high-storied

buildings are prepared by the CAD system. In this figure, the color buildings indicate the buildings prepared by the

316


CAD system. Fig. 5 shows the surface mesh based on the triangular element. Fig.6 shows the finite element mesh

based on the tetrahedral element. The automatic mesh generation method based on Delaunay method (Taniguchi and

Yamashita, 2001, Takada and Kashiyama, 2008) is employed. The fine mesh is employed near the ground and

buildings. The total number of nodes and elements were 2,458,388 and 14,115,104 respectively.

In order to find the bent element, the evaluation of the quality of mesh idealization is performed by using the

criterion of mesh quality (Freitag and Knupp, 2002). Fig. 7 shows the distribution of mesh quality. The blue collar

denotes the good quality and the red collar denotes the bad quality. Fig.8 shows the seen verifying the quality of

mesh idealization. The observer can see the image in detail from the arbitrary view point�and can share the image in

VR space with other observers at same time.

FIG.3: Configuration of numerical wind tunnel

FIG.4: 3D shape model

�

FIG. 5: Surface mesh

317


FIG.6: Finite element mesh

FIG.7: Distribution of mesh quality�

FIG.8: The seen verifying the mesh quality

4. WIND FLOW SIMULATION Assuming the viscous incompressible flow in Newtonian fluids, the wind flow can be described by the Navier-

Stokes equation based on Boussinesq approximation.

(1)

(2)

0)(1

,,,, =+−++ijjjjiijiji

uupuuu νρ

0=i,i

u

318


where iu is the velocity, p is the pressure, ρ is the density, ν is the viscosity of fluid, respectively.

For the boundary conditions, the following boundary conditions are employed.

� �iiuu =

iijjiijj huupn =++− ))(( ,,νδ

where ijδ is the Kronecker’s delta,

jn is the outward unit normal to the boundary.

For the discretization in space, the stabilized finite element method based on the SUPG/PSPG�(Tezduyaer 1991,

Kashiyama et al. 2005)�is employed for the spatial discretization of Navier-Stokes equation. For the discretization in

space, the linear interpolation for both velocity and pressure based on tetrahedron element (P1/P1 finite element) is

employed. On the other hand, the Crank-Nicolson scheme is employed for the discretization in time. To solve the

simultaneous equation, the element-by-element Bi-CGSTAB2 method is applied.

A parallel computational scheme based on the domain decomposition method (Kashiyama et al. 2005) is employed

in order to reduce the CPU time and computer storage required. A parallel implementation using the MPI suitable

for unstructured grid is designed for the use on PC cluster parallel computer. For each sub-domain, the processor

associated with that sub-domain carries out computations independently.�The details of the discretization and

parallel computation have been shown in the reference (Kashiyama et al. 2005).

The simulation method is applied to the wind flow simulation in urban area, Tokyo, Japan. The no-slip boundary

condition is applied to the surface of landform and buildings. The slip boundary condition is employed on the side

wall. The computed velocity and pressure at nodal points are stored in the data file for flow visualization at every 1

second. Fig. 7 shows the computed streamline which is visualized by the commercial visualization software AVS

(Application Visualization Software). From this figure, it can be seen that the wind flow passed large buildings

shows the complicated flow field.

FIG.9: Computed stream-line

5. POST-PROCESSING USING VR TEQUNIQUE

For the visualization of numerical results based on 3D stereo immersion, the commercial visualization software

AVS Express MPE is employed to know the macroscopic flow phenomena (see Fig.9). On the other hand, in order

to know the microscopic flow phenomena, the open software VFIVE (Vector Field Interactive Visualization

Environment; Kageyama et al., 2000, Ohno and Kageyama, 2007), which is a general purpose visualization software

for the CAVE and is written by VR (Open GL and CAVE library) programming, is employed. VFIVE interactively

shows following visualization objects in the VR space; field line, particle tracer, vector arrows, iso-surfaces, etc. The

major visualization methods of VFIVE are listed in Table 1. The user can choose the visualization method by using

(4)

(3) on g

Γ

on h

Γ

319


a small portable controller called wand in VR space (see Fig.10). Fig. 11 shows the seen verifying the three

dimensional flow field. The observer can understand the microscopic flow field interactively by using VFIVE.

TAB. 1: Major visualization methods of VFIVE

FIG.10: Menu of visualization method in VR space (for velocity field)

�

FIG.11: Visualization by stream lines and vector arrows

320


FIG.12: Visualization process of Fusion VR

The attempt to combine the different CG images created by different visualization software is performed. Fig.12

shows the visualization process of the software, Fusion VR (Miyachi et al. 2005). This software realize that the CG

images created by different software are captured and the composite image is produced in Master-PC, and the

information of composite image is transferred to each Slave-PC. Fig. 13 shows the image of composition of the CG

images created by AVS Express MPE (stream lines) and the VFIVE (vector arrows). Using this type of technology,

the observer can understand both macroscopic and microscopic flow phenomena easily at same time.

FIG.13: Visualization of macro- and microscopic velocity field

321


6. CONCLUSIONS

The application of the virtual reality technique to pre- and post processing for wind flow in urban area has been

investigated in this paper. An accurate modeling method using CAD and GIS data and an automatic mesh generation

method have been developed. The visualization method based on VR technique has been employed for both pre- and

post-processing to understand the quality of mesh and flow field. The attempt to combine the different CG images

created by different visualization software has also been performed. From the application in this paper, it can be

concluded that the present method using VR technique is shown to be a useful tool to investigate the wind flow in

urban area at the stage of planning and designing.

7. REFERENCES

Takada, T.�and Kashiyama, K. (2008), “Development of an accurate urban modeling system using CAD/GIS data

for� atmosphere environmental simulation”, Tsinghaua Science and Technology, Vol.13, No.S1 (Proc.

ICCCBE-XII&INCITE 200), pp.412-417.

Freitag, L.E. and Knupp, P.M. (2002), “Tetrahedral mesh improvement via optimization of the element

conditionnumber”, Int. J. Numer. Meth. Eng., Vol.53, pp.1377-1391.

Tezduyar, T.E.�(1991), “Stabilized finite element formulations for incompressible flow computations”, Advances in

Applied Mechanics, Vol.28, pp.1-44.

Kashiyama, K., Shimizu, T., Kurahashi, T. and Hamada, H.�(2005). “Parallel finite element simulation for

orographic wind flow and rainfall using GIS data”, Int. J. Numer. Methods in Fluids, Vol.47, pp.575-589.

AVS Express MPE: http://www.kgt.co.jp/english/products/mpe/index.html

Kageyama, A., Tamura, Y. and Sato, T. (2000), “Visualization of vector field by virtual reality”, Progress of

Theoretical Physics Supplement, No.138, pp.665-673.

Ohno, N. and Kageyama, A. (2007),” Scientific Visualization of Geophysical Simulation Data by the CAVE VR

System with Volume Rendering”, Phys. Earth Planet. Interiors, Vol.163, pp.305-311.

Miyachi, H., Oshima, M., Ohyoshi, Y., Matsuo, T., Tanimae, T. and Oshima, N. (2005), “Visualization PSE for

multi-physics analysis by using OpenGL API fusion technique, Proceedings of the First International

Conference on e-Science and Grid, pp.530-535.

322


CONSTRUCTION PROCESS SIMULATION BASED ON SIGNIFICANT DAY-TO-DAY DATA

Hans-Joachim Bargstädt, Prof. Dr.-Ing. M.Sc.,

Bauhaus-University Weimar Germany;

[email protected], www.uni-weimar.de

Karin Ailland, Dipl.-Ing.,

Bauhaus-University Weimar Germany;

[email protected], www.uni-weimar.de

ABSTRACT: Everyday life on construction sites is commonly characterized by enormous pressure due to time and costs as well

as difficult logistical requirements. Information Technologies offer great potential to solve these problems. These

technologies are to be used more intensively to find new creative and efficient methods of construction.

Modern simulation tools can be applied with increasing success. Until this point, these tools for the optimization of

construction and logistic processes have predominantly been used only in the start-up phase of a project. However,

projects are often affected by unscheduled constraints and limitations that give reason to deviate from the formerly

optimized plan and to find ad-hoc solutions, especially in the erection phase.

In order to meet these requirements, the application of simulation tools from the planning phase can be extended to

the erection phase. However, a more specific database is needed for this.

The topic of this paper has been developed in response to the question of what the capabilities of using simulation

tools are, and how they might be able to incorporate the available information about the actual status of

construction to improve progress tracking and also enhance dynamic scheduling.

This paper outlines an approach based on accurate day-to-day data for the current project state at any time. These

data then facilitate the simulation of possible variations for ongoing optimization. Thus, the critical path monitoring

and the flexibility in case of changes will be improved immensely. Long-term consequences will be recognized at an

earlier stage.

The basis for accurate data pooling is an efficient and automated survey. This yields two challenges. First, it is

necessary to determine which choice of data is significant and actually needed for evaluating the day-to-day status

in construction progress. Secondly, the required data must be captured as efficiently as possible during the ongoing

working activities.

KEYWORDS: Simulation, optimization, process monitoring, evaluation techniques, day-to-day data

1. INTRODUCTION

Non-stationary construction processes are commonly characterized by a great number of failures and changing

boundary conditions as well as enormous time and cost pressure. In order to ensure the efficient use of valuable

resources in spite of these challenges, a scheduling technique is needed that allows for active control and steering.

Planning methods that feature adequate adaptability and support the description of parallel processes, unexpected

faults, and stochastic and fuzzy parameters are therefore necessary (Hohmann, 1997).

Thus modern simulation tools can be applied with increasing success. Until this point, these tools for the

optimization of construction and logistical processes are predominantly used in the start-up phase of a project.

However, projects are often affected by unscheduled constraints and limitations, especially in the erection phase,

that give reason to deviate from the formerly optimized plan and to find better current solutions.

The concept presented in this paper was developed with particular attention to time control. Experience in this field

demonstrates that bar diagrams are still the predominant tools on most construction sites. Even critical path methods

323


are seldom used. Simulations, however, can provide much more information, such as the machine’s productivity,

possible hidden capacities, and reasons for reduced output (Hohmann, 1997).

In order to meet the aforementioned requirements of daily construction business, the application of simulation tools

could be extended from the planning phase to the erection phase. A more specific and dynamic data base is needed

to effectively address this challenge.

2. SIMULATION

Most research approaches that deal with process modelling in construction management strive for the support of

decision-making processes. Therefore, linear and nonlinear optimization models that implicate relevant influences

and boundary conditions are used,. In addition to these, simulation tools are used more and more intensively for

decision-making. Even if the results are not optimal, they provide appropriate solutions for complex problems. The

use of simulation tools becomes increasingly attractive as processes become more complex with multiple influences

that must be considered. In simulation the mathematical complexity can be kept at a lower level compared to the

mathematical linear and nonlinear optimization (Tecnomatix, 2006).

Other branches of industry such as automobile (Bayer, 2003) or shipbuilding (Beißert et al, 2007) already use

simulation with great success. Examples include the planning of assembly lines and logistical processes.

Methods used in these related areas must now be transferred from stationary industries to the building industry. In

comparison the building industry is distinguished by individual products that vary in construction type, size,

function, material and many more attributes. Moreover the construction conditions change with every new project as

well as within the execution phases of the project. And within a construction site the working places change and

move as the construction process progresses.

The authors use event-based simulation, as in Tecnomatix plant simulation (Tecnomatix, 2006), which is already

successfully utilized in stationary industry.

2.1 Strategies

An important challenge in simulation is how to keep the vast amount of possible solutions and variations to a

manageable number. Often the construction sequence is not clearly determined by technological constraints, but can

be varied to a high degree. This degree of freedom is synonymous with a lack of clearly defined objectives in a

project. The better the objectives for a project are defined, the more easily the final binding conditions can be

derived. Specifying parameters can be introduced by the investor as well as by the design engineers, construction

companies, or any other parties involved in the whole project.

Concerning the example given in this paper, the vast possibilities of influencing parameters by parties other than the

construction company shall not be considered. In our focus on the construction company, we also assume that the

construction project is not exposed to any initial limiting conditions. Thus a high degree of freedom is given in terms

of how to manage the construction process and how to choose feasible sequences.

In terms of simulation tools, this freedom makes it possible to search not only for an optimum given by past

experience, but also to include possible other and new sequences. At this point experienced site managers might

argue, that, following this approach, most of the randomly generated sequences and dependencies would need to be

checked by hand to determine whether they can be realized in reality and how they can be valued in terms of time

and resources.

One approach to better find appropriate solutions is the definition of certain construction strategies. They are

derived from past experience, but then described in a generalized manner. They can be used to check on the

improvement of solutions from simulation.

Two alternatives for defining construction strategies are possible. The first alternative is the standardization of

proven patterns from former construction projects. This can be considered as a tool kit, including standardized and

detailed processes, which are then included as subroutines within a bigger set of simulation frames. This works like

a library of simulation subroutines with sets of parameters in order to fit them into the specific simulation model.

324


The second alternative is the definition of main features of construction strategies. This aims to use simple and

transparent guiding assumptions in order to enforce certain clear sequences within a complex project. Since they are

also determined through experience, they are easy to understand and can also be easily checked.

Even complex construction processes yield many possible basic strategies. Starting with the construction sequence,

a certain pattern (“from left to right”, “from bottom to top”, “along a certain axis”, etc.) can be defined, which then

governs all of the following work steps. Other strategic base lines can be “from coarse to fine”, “first inner, then

outer structures”, or “destruction before construction”.

Other conditions are more likely to be characterized as arguments for favorable priorities. This makes it possible to

distinguish between activities of short and long duration. It enables differentiation between dirty and clean activities,

between wet and dry processes, etc. Concerning the analysis of sequences on real construction sites, often one or

two of these patterns are the main characteristics of the favorable solution (Bargstädt, 2008).

For the successful implementation and reasonable choice of possible execution strategies, however, it is necessary to

appropriately identify the actual building progress. The progress should be described as exactly as necessary and as

efficiently as possible.

3. DESCRIPTION OF THE ON-SITE STATUS

The answer to the question “Which data do we really need?” is going to be a little more complex. The first aspect to

consider in this context is time control. We need reliable information about degrees of completion and the times of

erection - more detailed information about the processes and sub-processes of the project.

There are different ways to describe the building progress. For example, many research projects draw conclusions

from geometry, by analysing image data or scatter plots. Another source of data is the financial progress. For

example, the currently used reporting system is predominantly made for capturing all information needed for

controlling costs.

With regard to time control, a method will be developed that accurately represents the daily building progress by

identifying single-process steps. This method will be sufficient for time control and more cost-efficient than

methods that capture geometric data, because there is less need for expensive technologies, and the analysis will be

less time consuming.

This method starts from a process model that represents all process steps, dependencies, and supporting documents,

such as delivery notes, site reports, quality checks and acceptance certificates.

The curb of a bridge has been chosen as an example because it represents an almost independent, but typical self-

contained element of reinforced concrete construction. Therefore, the findings are likely to be transferable to other

construction processes, since reinforced concrete is a widely used building material.

3.1 Building process

The process of constructing bridge curbs is not thoroughly described in the literature. Therefore the building process

of several bridges was monitored, captured in detail, and

documented. Different road bridges with varying degrees of

complexity and different constraints were selected. In the

following section, one of these projects is described in detail.

This largest of all monitored bridges is a 445 m long highway

bridge in Germany with 12 spans and a double-webbed t-beam

cross-section made of pre-stressed concrete. This bridge was

built with mobile scaffolding. The formwork for the curb was on

the ground bridge slab (Figure 1).

This bridge shows multifaceted boundary conditions that

strongly influence the building process and methods as well as

the time scheduling. The bridge spans two farm tracks, an

interstate highway, a highly frequented main railroad line, and a

FIG. 1: Bridge curb (Bundesministerium, 2004)

325


river. Thus all disturbances, including blocking the railroad track, had to be kept to a minimum. The river is part of a

drinking water protection area and is subject to strict environmental regulations.

Right from the beginning of the execution the construction sequence deviated and had to be constantly rescheduled

by the site manager. The ongoing direction of the building progress was based solely on the site manager’s

competence. He had to act on a day-to-day basis. A thorough consideration of long-term consequences was almost

impossible.

Within the monitoring of the real process, all interferences and the following deviations as well as the corrective

activities were documented in order to use them for the presented approach as well as for future simulation

scenarios. Furthermore, the duration of all activities, as well as the exact need for material, machines, and manpower

was monitored.

Other bridges were also monitored in this way in order to gather a reliable data source. Based on these data a

universally valid and conventionalized construction process for a bridge curb was designed. Event-driven process

chains (EPC) were used to analyze and visualize all dependencies within the building process, because of their easy

application and transparency (Figure 2).

The construction process is structured in horizontal and vertical directions (Buchhop, 2007). In the horizontal

direction it is structured in terms of main, basic, sub, and manufacturing processes. Furthermore, it is structured in

FIG. 2: Construction process of a bridge curb

326


terms of functions, events, and logical and causal dependency relationships. Every function starts with an event, for

example a certain activity or a special point in time. These events are identified and described exactly.

The vertical direction defines the degree of detail according to which a process is structured. For example one

structural element can be considered as one piece or with all of its single elements. It is defined by the requirements

of the simulation. Thus the processes are structured as detailed as is reasonable in order to implement building

strategies and to identify realizable corrective actions in case of delays.

After structuring, all documents coming along with the building process were monitored along with their start time

and duration. Furthermore, all documents were attached to process events.

Finally all process times were monitored as durations of activities as well as cycle times for recurrent processes

(minimum, maximum, and average).

3.2 Evaluation of sub-processes

When looking at a detailed construction process, sub-processes and activities are evaluated in terms of their

importance and significance in relation to the aggregated process.

The idea in structuring the building process is to identify “grid points”. This term is adopted in analogy to analytical

algebra. Using this approach, a curve’s shape can be defined if a sufficient number of discrete points and the type of

curve are known. In the same way a sufficient amount of reliable process information will be used to define the

actual status of work. This information will be taken as grid points for the indication of the actual state of

construction progress. Grid points are set up as control gates from which signal information can be generated or

taken. They indicate clearly, or at least with sufficient certainty, which events have taken place and which activities

are in progress. Therefore, the kind and density of required information is to be determined.

As already mentioned, each process starts with an event, which can be a certain activity for example. These events

have to be identified and described as grid points. Appropriate grid points are, for example, events required before a

certain procedure can start (material delivery), events assigned to the completion of a procedure (milestone), easy-

to-describe intermediate states (finishing of sealing or acceptance of reinforcement), and placement of certain types

of rebar and installation units. Grid points are comparable to a less stringent type of milestone in time schedules.

The identification of the most significant sub-processes is important. The AHP (Analytic Hierarchy Process) method

is used to rank the processes by the mentioned criteria order to identify key processes, which are of particular

importance for the ongoing optimization. Later these processes are identified by significant events in the

construction process. Sensitivity analysis is implemented to check the plausibility of the criteria and their emphasis

in the AHP.

The specification of criteria and their scales are only briefly mentioned here due to their complexity. Examples of

criteria are kind and number of dependencies to other processes, process duration, required resources, fault liability,

error probability, flexibility of execution order and date with respect to parallel processes, and possible correction

strategies etc.

For the criteria, varying scales with values based on deterministic and stochastic concepts are defined. In order to

determine those values, different methods will be used. For example, the flexibility of execution order and date will

be determined by using the Critical Path Method (CPM), so deterministic values will be calculated. Another

example is the process duration. A realistic description of the duration is very important for an appropriate

simulation, but it depends very much on the boundary conditions. Therefore stochastic methods as fuzzy logic have

to be used. Finally, a sensitivity analysis is implemented to check the plausibility of the criteria and their emphasis in

the AHP. Aside from the identification of significant events, the results will be used for the validation of simulation

strategies as mentioned earlier.

Next, the grid points have to be connected with the real building process by progress data.

4. DATA SOURCES

In discussing the available data sources, an evaluation of the usable instruments must be completed. Up to this point

most research projects develop and analyze data gathered from one type of source, for example RFID. But the

327


reality in construction varies so much that different techniques of data gathering have to be combined. Thus an

appropriate mix of these techniques will be established within the ongoing research project. Therefore, different

sources of information are identified parallel to the process modeling that in total give significant information about

the current state of the construction (Bargstädt, 2008).

4.1 Documents as information sources

The approach to designing an efficient and realistic controlling tool starts with the identification of documents and

other information sources that are widely used on most construction sites. Documents like construction records,

minutes of meetings, or delivery notes are associated with the everyday life on each construction site. Up to this

point, however, they have rarely been evaluated in terms of capturing accurate information about the building

progress daily. These documents are analyzed with reference to the information they yield with respect to the degree

of completion or material deliveries. Signal information as described above is a matter of particular interest.

Furthermore, the common data format is registered, which in many cases is handwriting. Typical information

includes:

• deployment/staff data such as number of workers, qualifications, working hours (start/end) and

completed work

• machine data such as machine inventory, working hours/place, and quantities completed

• supply data such as shipments

• test results

• acceptance results

• disengaging dates

• concreting times (start/end)

• site measuring quantities

• general output documentation (description of daily output)

• weather data such as temperature, hours of rain, and wind speed

To extract this information, the authors started researching the question of which documents are commonly

generated on construction sites. This research was based on the literature and on a nationwide survey addressed to

site mangers who work for construction companies of all sizes.

The site documents can be divided into three groups: documentation on a normative basis, on a contractual basis,

and for internal controlling issues. These documents are evaluated in terms of the information they provide about the

construction progress. This information is assigned to categories as shown in Table 1.

Table 1: Information categories

categories Data type Definition

resources supply data Documentation of material entries.

stuff data Documentation of number and qualification of workers. capacities

↓

production output machine data Documentation of number and type of machines.

site measuring data Data for billing, documentation of quantities and work

completed.

production output

testing /

disengaging data

Data for technical control, state of completion, and limits of

quality.

production influences Progress documentation Keeping evidence. Documents for possible court cases, for

example as correspondence.

production conditions Weather data Documentation of production conditions.

328


By defining and emphasizing different criteria, the documents’ significance is evaluated and made comparable. The

following criteria and objects are defined and emphasized on simple ordinal scales (Table 2). The evaluation is made

according to AHP. In a subsequent step its applicability for digitization and automation is discussed.

Table 2: Criteria for document/information evaluation

Criterion Explanation

significance Answers the general question of whether the information is significant to conclude on the

status of the building progress. Knock-out criterion.

reference What can the information refer to?

To a specific activity, structural element, working section, or to the construction site as a

whole.

degree of completion To which degree of completion can a conclusion be reached from the information

provided?

To 100% completion, to accurate progress status, or to predefined intermediate stages?

gathering rhythm How often is the information taken?

When the event takes place: every day, weekly, monthly, not predictably.

gathering probability Is the information reliable?

content type Does the document contain factual or explanatory information?

conclusion type Is it possible to draw a direct conclusion?

As expected, all kinds of testing protocols have a high significance with regard to the identification of construction

progress. Additional staff and machine data can be used for plausibility control, since this information must be

combined with further facts for a reliable conclusion to the construction progress.

Data pooling based on the above-mentioned information sources does not yet describe the building process as

accurately as needed. On the one hand it is handwritten and often biased; on the other hand it is not yet sufficient for

a complete picture of the construction status. Depending on the intended optimization, further information must be

captured. Therefore, more sophisticated instruments for data evaluation are considered. It must also be kept in mind

that additional information sources should not be based on personal observation and should be protected from errors

and bias.

4.2 Instruments for data evaluation

Current technical instruments include RFID (Ergen, 2007), GPS, Photogrammetry, Laser scanning, Tachymetry, and

Bar codes (Chen, 2002).

In all of these fields further improvements will be realized soon. As a result of developments in most technologies it

is possible to push the boundaries that were limiting the applications of these technologies to construction just a few

years ago. An example is RFID technology, where the size of a RFID chip has become one-tenth of its original size

within 5 years, and the price has dropped to 1/100 of the original.

Furthermore, increasingly powerful screening machines create new possibilities for improved application. For ex-

ample the storage capacity of Photogrammetry increased enormously, and it is possible to exploit many more data

points in less time. It even seems realistic that real-time feedback will be possible in the near future (Kutterer, 2006).

In addition to these developments, the exchange and storage of data via the internet has created new possibilities.

Nowadays it is possible to transfer huge amounts of data to any place in the world, to process them in a central

location, and to transfer them back to site. The task is to identify the relevant evaluation instruments and register and

analyze them with regard to their ability to record real-time data. This comprises speed, data volume, precision, and

automation of data. It leads to the question of which instruments are appropriate to recognize parameters relevant for

the building progress. Similar to the document evaluation, constraints and requirements needed for the application of

329


different instruments are considered. Significant parameters are functionality, type of information, degree of

automation, and type of equipment.

Based on the above examination, the different information sources and instruments are methodically compared to

one another and interpreted with regard to a specific subsequent processing within the system for capturing the real-

time state of construction.

The possible information sources and instruments are evaluated with regard to the construction process to be

controlled. They are systematized and tested on their ability to capture signal information at grid points, both as

single sources and when combined with other types of sources. In addition, a catalogue of typical construction

elements is developed, which will answer the question of how best to capture different kinds of elements.

The instruments are compared based on multiple criteria including reliability (subjective or objective), quality,

flexibility, workability, complexity, comparability with standard values, etc.

5. COMBINING PROCESS MODELLING AND DATA EVALUATION

The basics developed so far concerning process modelling and data evaluation will now be combined into one

model. By using process modelling, the building progress is structured in events, functions, relations, and additional

grid points pursuant to the requirements of deadline monitoring and simulation. Thus, it defines where and when

information should be captured.

Now those points will be matched with the possible data sets as shown in Figure 3. Preference is given to real-time

data. In the first step the model will be fed with data that are taken according to availability and actuality

(information sources). These data are in any case reviewed with regard to their relevance and their ability to describe

the status at a considered grid point. Data with less relevance will be held back for further usability, for example

checking on the plausibility.

FIG. 3: Conception

330


Thus monitored, the virtual model of the building process is generated in some main elements, but may still be

incomplete. Now further appropriate instruments for data evaluation will be selected and activated in order to

complete the monitored situation. They will supply the monitored model with sufficient information for the

remaining grid points. This supplementary data gathering is taken by means, which are chosen based on the

aforementioned multi-criteria comparison.

Again the question of which instruments will be appropriate to capture the necessary information at certain grid

points has to be answered. At first glance almost all instruments for data gathering have the potential for a large

quantity of accurate data, as long as time and costs don’t have to be considered. But in practice neither of these

resources are unlimited. It is also not adequate to extrapolate the further development of the tools to infinity.

For the present time and status a catalogue is established with classifications, that point out which instruments are

appropriate to best describe different significant construction status. Nevertheless the opportunities to increase the

reliability and significance by capturing more data, by using more sophisticated instruments, or by referring to

recent developments in monitoring instruments are also considered.

Using a sample construction process as a test environment, a complete simulation model with different data

evaluation and grid points is developed. On this model, tests are run with regard to data formats, evaluation

intervals, and response times. An important aspect is the fast and integrated data handling, which allows daily

steering. In everyday construction business, this time relation is of particular importance for an accurate daily

simulation. Of course, the automatic generation and evaluation of the data is one special aspect. Another important

aspect is to make use of combinations of data and of redundancies for plausibility checks. At the same time

redundant information may also detect contradictory data, which calls for special algorithms to distinguish between

true and false data. The individual adjustment of the different information sources and instruments is essential for

the efficiency of the whole monitoring system.

6. CONCLUSION/PERSPECTIVES

One objective of this research is to evaluate applications and limits of the presented concept using grid points. A

further aspect is the question of how the evaluated data can be implemented in a simulation tool. Using simulation in

construction processes that have been developed at the Chair of Construction Engineering and Management at

Bauhaus-University Weimar (Beißert et al, 2007), the prototype implementation of different instruments for data

generation will be added to the simulation model. The effects of different performances then will be analyzed.

Simulation models are suitable for developing construction sequences, as shown in (König, 2007). This approach

considers the monitoring aspects of construction sites. Data gathering, data processing, and the following steering

processes are implemented in the simulation model. It yields information about the importance of precision,

velocity, automation, redundancy, and significance of data capturing during construction processes. The results,

advantages, disadvantages, weak points, and application limits can be adjusted for different construction conditions.

Furthermore, the error-proneness can be tested. One important aspect of this will be missing data and how the

simulation will react in the case of uncertainties or, for example, a missing delivery note.

With the knowledge gained from these simulation experiments, the results will be verified on a prototype. Even if

the model suggests complete control of the construction process, the reader should be aware that construction is far

too complex to rely simply on automation. The presented approach is a way to improve the information on ongoing

construction sites. It will enable the construction manager to better concentrate on real obstacles that have not been

encountered in advance, and to have better information sources for routine processes to be steered. Thus the time to

react will be shortened. It is then also possible to evaluate effective counteractions by using the simulation tool in

combination with real-time data gathering.

331


7. REFERENCES

Bargstädt H.-J., Ailland K. (2008), “Capturing the real time state of construction for simulation”,

IABSE Conference, Helsinki, Finland, Conf. Proceedings.

Bayer J. (2003), „Simulation in der Automobilproduktion“, Springer, Berlin.

Beißert U., König M., Steinhauer D., Bargstädt, H-J. (2007), “Constraint-Based Simulation of Outfitting Processes

in Shipbuilding and Civil Engineering”, 6th EUROSIM Congress on Modelling and Simulation, Conf.

Proceedings, Ljubljana, Slovenia.

Buchhop E. (2007), „Zeitliche Erfassung von Kernprozessen als Teil der Prozessanalyse“, bdvb-Award

Geschäftsprozess- und Projektmanagement, CT Salzwasser-Verlag, Bremen.

Bundesministerium für Verkehr, Bau- und Wohnungswesen (2004), „Richtzeichnungen für Ingenieurbauten“,

Verkehrsblatt-Verlag, Dortmund.

Chen Z. H. Li, Wong C.T.C. (200x), “An application of bar-code system for reducing construction wastes”,

Automation in Construction, Vol. 11, no. 5, pp. 521–533.

Ergen E., Akinci B., Sacks R. (2007), “Tracking and locating components in a precast storage yard utilizing radio

frequency identification technology and GPS”, Automation in construction, Vol. 16, pp. 354–367,

25.07.2006.

Hohmann G. (1997). „Von der Netzplantechnik zur Simulation - Analyse von Bauprozessen mit Hilfe von Petri-

Netzen.“ Internationales Kolloquium über Anwendungen der Informatik und Mathematik in Architektur und

Bauwesen , IKM , 14 , 1997 , Bauhaus-Universität Weimar, Germany.

König M., Beißert U., Bargstädt H-J. (2007), “Constraint-Based Simulation of Outfitting Processes in Building

Engineering”, 24th W78 Conference, Conf. Proceedings pp.491-497, Maribor, Slovenia.

Kutterer H., Hesse C. (2006), “High-speed laser scanning for near real-time monitoring of structural deformations”,

Tregoning, P., Rizos, R. (Eds.): Dynamic Planet – Monitoring and Understanding a Dynamic Planet with

Geodetic and Oceanographic Tools, IAG Symposia, Vol. 130, Springer, 776 - 781.

Tecnomatix (2006). “Tecnomatix Plant Simulation, Step-by-step Help”, Handbook Version 8.1., Stuttgart.

332


EFFECTIVENESS OF SIMULATION-BASED OPERATOR TRAINING

John C. Hildreth, Assistant Professor,

The University of North Carolina at Charlotte;

[email protected]

Michael Stec,

Volvo Construction Equipment North America, Inc.;

[email protected]

ABSTRACT: Construction equipment manufacturers provide on-machine operator training, consisting of individual

instruction regarding general operational performance and machine specific controls. Such training is conducted

in the field using the subject equipment to provide field operational experience. Construction equipment simulators

present an opportunity to train operators for stressful and tough construction environments without needing to

employ an actual machine. Eliminating machine use saves fuel, reduces equipment wear, and lessens the inherent

risks of damage to machine and man.

This study was performed to evaluate the effectiveness of full-motion and zero-motion simulation-based training

using the Volvo Advanced Training Simulator (ATS) relative to on-machine training. Operational performance

before and after training was measured in terms of loading cycle time and production rate. Levels of operator

confidence and anxiety were also collected. Results indicate that full-motion simulation-based training increases

production rate and confidence, while decreasing cycle time and anxiety.

KEYWORDS: Construction Equipment, Simulation, Training, Operator Training.

1. INTRODUCTION

Construction equipment manufacturers are called upon by the industry to provide training for equipment operators.

For Volvo Construction Equipment North America (Volvo), such training is typically on-machine, consisting of

personnel providing individual instruction regarding general operational performance and machine specific controls.

Instructional topics include familiarization with equipment controls, maneuvering the equipment, operational

efficiency, and operational safety. Training is conducted in the field using the subject equipment to provide field

operational experience.

Construction equipment simulators present an opportunity to train operators for stressful and tough construction

environments without needing to employ an actual machine. Eliminating machine use saves fuel, reduces

equipment wear, and lessens the inherent risks of damage to machine and man. Equipment simulation allows

operators to quickly and efficiently be introduced to the equipment and its operation. The benefits of simulation-

based training are perceived to be increased training efficiency and improved operator skills.

Volvo has developed an Advanced Training Simulator (ATS) of an L120F wheel loader. The ATS is a full-motion

simulator designed to aid in training wheel loader operators. The ATS consists of a L120F loader cab mounted on a

hydraulically activated platform with six degrees of freedom, the training simulation software and computer, and an

imaging system to display the three dimensional graphics.

Volvo and The University of North Carolina at Charlotte designed and conducted a research project to evaluate the:

• improvements in operator skills realized through simulation-based training;

• effect of simulation-based operator training on operator anxiety and confidence levels;

• effect of full-motion simulation; and

• feasibility of simulation-based operator training.

The purpose of this study was to:

333


1. aid in understanding proper application of the ATS in developing fundamental skills in operators

possessing no previous loader experience;

2. provide preliminary data regarding improvements realized from simulation-based operator training in terms

of loading cycle times and productivity;

3. provide preliminary data regarding the impact of simulation-based training on operator anxiety and

confidence levels; and

4. provide preliminary data comparing and contrasting full-motion simulation, zero-motion simulation, and

on-machine operator training.

2. BACKGROUND

Simulators are used to provide a safe and inexpensive practice environment for persons who operate complex

machines such as airplanes, trucks, cars, and construction equipment. Simulators are particularly useful to

researchers investigating problems in which the real environment is too hazardous, costly, or difficult to control.

Simulation has been widely applied to the vehicle driving environment to explore the effects of cell phones on

driving performance (Schneider and Kiesler 2005), conditions that lead to better in-vehicle performance (Bullough

and Rea 2001), and devices to help mitigate accidents with in-vehicle warning systems (Enriquez et al. 2001).

Studies have established correlations between participant behavior in driving simulators and behavior in real

vehicles (Godley et al. 2002, Panerai et al. 2001, Tornros 1998).

Simulators are effective training tools because of their ability to replicate a real environment and the realistic

behavior of simulation participants. Two recent studies regarding the use of simulators in training snowplow

operators have been sponsored by state departments of transportation (Strayer et al. 2004, Mashiocchi et al. 2006).

In both reports, simulation-based training was well received by participants and found applicable to operators at all

levels of experience.

Simulation-based training has also been applied to training of construction equipment operators. Wang and Dunston

(2005) present a survey of advanced virtual training technologies for training heavy construction equipment

operators. Gokhale (1995) describes the development of a simulator for training crews to operate the JT2510

directional drilling machine. The objectives of the simulator use were to familiarize operators with the controls,

train the operators to steer the machine, and enable evaluation of operator performance.

The effectiveness of simulation-based training for construction equipment operators is not well documented in the

literature. A multifunctional simulator for training operators of haul trucks, excavators, and surface drill rigs in a

mining application has resulted in improved operator performance and reduced operational damage to equipment

(Anon. 2005). A single study quantitatively documenting the effect of operator skill training was found (Kamezaki

et al. 2008). Participants took part in simulation-based training sessions for double front work machines over a 3

day period. Operator skills in terms of task completion time and positional accuracy were significantly improved.

3. METHODOLOGY

A total of 17 UNC Charlotte students volunteered to participate in the study. Two identical study sessions were

conducted, with 8 students participating in the first session and 9 in the second. The requirements for eligibility to

participate were that students must have no previous experience with operating a wheel loader and be physically

capable of operating a wheel loader. Training sessions were conducted at the Volvo Customer and Demonstration

Center in Asheville, North Carolina and consisted of an information briefing, pre-training testing, operator training,

and post-training testing.

3.1 Informational Briefing

Participants received an approximately 15 minute briefing describing the study and basic wheel loader operation.

Each participant received an information packet containing a description of the study, written pre-test, written post-

test, and wheel loader operating guidelines and tips. During the briefing, participants were randomly and evenly

assigned to each of three operator training methods: full-motion simulation, zero-motion simulation, or on-machine

training.

334


3.2 Pre-Training Testing

Participants performed an operational pre-test to quantify their baseline operator skills and completed a written pre-

test to document their anxiety and confidence levels. During the operational test, participants operated an L220F

wheel loader to three pass load A35D articulated haulers from a material stockpile. Prior to the start of the

operational test, a “loader coach” briefed participants regarding the loader controls and truck loading operation.

Each participant was given 3.5 minutes to load each of the three articulated haulers. Because all participants had no

previous wheel loader operating experience, the loader coach was present in the cab during the loading of the first

truck, and subsequent truck loading operations were performed without the presence of the loader coach. The

operational tests were video recorded and the following data collected:

1. truck loading time (sec) – from stopwatch timing;

2. number of loader passes; and

3. payload of each truck (tons) – as weighed using industrial truck scales.

Immediately following the operational pre-test, each participant completed a written pre-test requiring them to

respond to questions regarding their levels of anxiety and confidence.

3.3 Operational Training

Participants received 25 minutes of operator training in one of three training methods: full-motion simulation, zero-

motion simulation, and on-machine training. Simulation-based training (full-motion and zero-motion) were

completed using the ATS to simulate use of the L120F to load A30E articulated haulers from a material stockpile.

Participants operated the simulated loader while Volvo personnel operated the simulation computer and provided

real-time feedback regarding their performance. In the full-motion simulation, the ATS was operated in the normal

mode with motion capabilities enabled. In the zero-motion simulation, the ATS was operated with the motion

capabilities disabled.

Participants completing the on-machine training used the L120F to load A30E articulate haulers from a material

stockpile at the Volvo Customer and Demonstration Center. The loader coach was present during training to

provide real-time feedback regarding performance.

3.4 Post-Training Testing

Participants completed a written post-test and then performed an operational post-test to quantify their operator

skills. The written post-test required each participant to rate their level of anxiety and confidence in operating a

wheel loader after training. The post-training operational test was conducted in the same manner as the pre-training

operational test. Each participant was given 3.5 minutes to load each of the three articulated haulers. The number of

loader passes for each truck and the loading cycle time based on stopwatch measurements were recorded. Each

truck was weighed using industrial truck scales and the payload of each truck was also recorded.

4. RESULTS AND DISCUSSION

Changes from pre- to post-testing in operational performance were calculated and expressed as a percent change

from the baseline (pre-training) values.

The measured loading time, number of passes, and payload data were used to calculate the production rate (tons/hr)

of the operation. Loading time and production rate data were statistically analyzed to investigate differences

between operator training methods.

Participants responded to questions regarding confidence and anxiety levels as part of the written pre- and post-tests.

Due to the categorical nature, a subjective analysis of the data was performed.

4.1 Loading Time

Loading time was the time required for the participant to load the truck and was measured in seconds using a

stopwatch. Due to the 3.5 minute time constraint for each load, some trucks were loaded with 2 passes and others

with 3 passes. To account for this disparity, a corrected loading time was determined by prorating the recorded

335


loading time to a 3 pass loading time. The corrected loading time was used for analysis and is presented here as the

loading time. It is clear from Fig. 1 that there was a decrease in loading time after training. Note that FM, OM, and

ZM are used to denote full-motion, zero-motion, and on machine training, respectively.

FIG. 1: Loading Time Box Plot

For each participant, the change in loading time was calculated as a percentage of the pre-training loading time.

This data is provided as Fig. 2, which shows a substantial decrease for all training methods. The greatest decreases

in average loading time were found in full-motion simulation and on-machine training, which were -33.50 percent

and -34.21 percent, respectively.

FIG. 2: Percent Change in Loading Time Box Plot

The percent change in loading time was analyzed for differences between the training methods. Proper analysis

required the data to be tested for normality and equal variances so that appropriate analysis methods could be used to

compare the data. Normality was tested using the Anderson-Darling test. All tests were performed at the 5 percent

336


confidence level (α=0.05). A resulting p-value less than 0.05 would indicate that the data exhibits a non-normal

distribution. A summary of the analyses is presented as Table 1.

TABLE 1: Statistical Summary of Percent Change in Loading Time

Based on the resulting p-values, the hypothesis of normally distributed data was accepted for each dataset. Bartlett’s

test for equal variance was used since the data was normally distributed. The resulting test statistic was 4.56 and

corresponding p-value was 0.102, indicating that the hypothesis of equal variances be accepted. The normally

distributed data of equal variances was appropriately analyzed using the analysis of variance (ANOVA) technique.

The results of the single factor (training method) ANOVA are summarized in Table 2.

TABLE 2: ANOVA in Percent Change in Loading Time

The ANOVA p-value was substantially greater than the 0.05 confidence level, indicating that the hypothesis of a

common distribution be accepted. This leads to the conclusion that training method does not significantly affect

changes in loading time.

While training method did not significantly influence changes in loading time, it was also important to determine

whether each training method resulted in a significant decrease in loading time. The Student’s t-test was selected as

the appropriate analysis method based on the previous determination of normally distributed data of equal variance.

The results of one-tail t-tests for decreased loading time are summarized in Table 3.

TABLE 3: t-Test Summary for Decreased Loading Time

The p-values for each of the three t-tests were substantially less than the 0.05 confidence level, indicating that the

hypothesis of no change should be rejected and a significant decrease in loading time be concluded.

4.2 Production Rate

The production rate calculated for this study is the rate at which participants would load material into trucks if trucks

were always available, in position, and loading operations were to occur continuously. In reality, this ideal situation

would not be achieved due to delays in the loading cycle resulting from waiting for trucks to come into position and

inherent inefficiencies in the loading operation. However, the calculated values are relevant given that the scope of

this study is limited to the operation of the wheel loader only. Production rate was calculated as the ratio of truck

payload to loading time.

337


It is important to note that the loading time used in the calculation is the actual loading time (not corrected loading

time) since the payload values were not also corrected. This calculation resulted in production rates in units of tons

per hour. A box plot of the production rates is provided as Fig. 3 and exhibits the pre- and post-training values for

each training method. From this figure, there is a clear increase in production rate after training.

FIG. 3: Production Rate Box Plot

For each participant, the change in production rate was calculated as a percentage of the pre-training production rate.

This data is provided as Fig. 4, which shows that production rates were substantially increased for all training

methods. The greatest increases in average production rate were found in zero-motion simulation and on-machine

training and were 77.06 percent and 74.37 percent, respectively.

FIG. 4: Percent Change in Production Rate Box Plot

The percent change in production rate was analyzed for differences between the training methods. The data was

tested for normality and equal variances at the 5 percent confidence level (α=0.05). Normality was tested using the

338


Anderson-Darling test. A resulting p-value less than 0.05 would indicate that the data exhibits a non-normal

distribution. A summary of the analyses is presented as Table 4.

Table 4: Statistical Summary of Percent Change in Production Rate

Based on the resulting p-values, the hypothesis of normally distributed data was rejected for zero-motion simulation

and on-machine training datasets. Therefore, Levene’s test for equal variance was used since the data was not found

to be normally distributed. The resulting test statistic was 0.07 and corresponding p-value was 0.931. This p-value

is substantially greater than the 0.05 confidence limit, indicating that the hypothesis of equal variances be accepted.

The non-parametric Kruskal-Wallis test was employed to analyze the non-normall distributed data with equal

variances. The results of the single factor (training method) Kruskal-Wallis test are summarized in Table 5.

Table 5: Kruskal-Wallis Test of Percent Change in Production Rate

The Kruskal-Wallis test p-value was greater than the 0.05 confidence level, indicating that the hypothesis of a

common distribution be accepted. This leads to the conclusion that training method does not significantly affect

changes in production rate.

While training method did not significantly influence changes in production rate, it was also important to determine

whether each training method resulted in a significant increase in production rate. The non-parametric Sign Rank

test was selected as the appropriate analysis method based on the previous determination of non-normally distributed

data of equal variance. The results of tests for increased production rate are summarized in Table 6.

Table 6: Sign Rank Test Summary for Increased Production Rate

The p-values for the tests were less than the 0.05 confidence level, indicating that the hypothesis of no change in

production rate be rejected and the conclusion drawn that all training methods result in an increased production rate.

4.3 Confidence and Anxiety

Participants ranked their levels of confidence and anxiety immediately prior to performing the operational tests by

responding to the following statements:

339


1. I am confident in my ability to effectively load trucks with a wheel loader.

2. I feel anxious when I think about loading trucks with a wheel loader.

Responses regarding confidence and anxiety were collected on a five point Likert Scale, assigned a numerical value,

and are presented as Fig. 5 and Fig. 6, respectively. There was a general increase in confidence levels from pre- to

post-training. Following training there was no disagreement regarding confidence, and nearly all responses

indicated agreement to strong agreement with confidence in effectively operating a wheel loader. It is clear that

operator confidence increased during the course of the study, but data is not sufficient to conclude that the training is

the cause. Rather, it is possible, and perhaps even likely, that the increased confidence is a result of the previously

inexperienced operators becoming familiar and more comfortable with the loader during the course of the study.

FIG. 5: Operator Confidence Box Plot

It is difficult to discern a general trend in the anxiety data. Prior to training, the complete range of responses were

given and mean values ranged from indicating neutrality to disagreement. Following training, no participant

strongly agreed with feeling anxiety and mean values decreased only slightly. The greatest reduction in anxiety was

exhibited by those receiving on-machine training, with the average post-training response being disagreement.

Interestingly, responses from the zero-motion simulation group remained unchanged. While the confidence data

appear to indicate that the participants became familiar and comfortable with operating the wheel loader, it does not

indicate that anxiety levels were substantially decreased.

340


FIG. 6: Operator Anxiety Box Plot

5. CONCLUSIONS

The purposes of this study were to aid in understanding proper application of the ATS and provide preliminary data

regarding the impact of simulation-based training and the potential for producing improved operator performance

relative to traditional on-machine training. Based on the results obtained during this study, it was concluded that

full-motion simulation-based training using the Volvo ATS is a viable option for training wheel loader operators.

There was no statistically significant difference found between on-machine and simulation-based training. All

training methods resulted in statistically significant decreases in loading time and increases in production rate.

The results indicate that trainees responded better to full-motion simulation training than zero-motion simulation.

Training using full-motion simulation resulted in decreased levels of anxiety, while the anxiety levels of participants

receiving zero-motion simulation remained unchanged. Full-motion simulation also resulted in increased operator

confidence.

It is recommended that a hybrid training method be developed consisting of full-motion simulation-based training

augmented with on-machine training. The objectives of this hybrid training method should be clearly defined in

terms of desirable operational parameters and the training explicitly designed to achieve the specific objectives. The

learning objectives of the on-machine portion of training may include proper bucket loading techniques and

comprehensive operational safety.

The scope of this study was limited to participants who were not previously experienced in operation wheel loaders,

and the results obtained pertain only to operators at the very origin of the learning curve. It is hypothesized that

additional differences in the training methods may become evident if participants with operating experience are

studied.

6. REFERENCES

Anon. (2005). “New Three-in-One Mining Equipment Simulator for Venetia.” Mining and Quarry World, 2(2), 46.

Bullough, J. D., and Rea, M. S. (2001), “Forward vehicular lighting and inclement weather conditions.” Proc.,

Progress in Automobile Lighting Symp. (PAL 2001), Darmstadt University of Technology, Germany, 79-89.

Enriquez, M., Afonin, O., Yager, B., and Maclean, K. (2001). “A pneumatic tactile alerting system for the driving

environment.” Proc., 2001 Workshop on Perceptive User Interfaces, Lake Buena Vista, FL, 1–7.

Godley, S. T., Triggs, T. J., and Fildes, B. N. (2002). “Driving simulator validation for speed research,” Accident

Analysis and Prevention, 34(5), 589–600.

341


Gokhale, S. (1995). “Partnering with Industry to Develop a Training Simulator for Directional Drilling Equipment

Operators.” Proc., 1995 ASEE Annual Conf., Anaheim, CA, 495-500.

Kamezaki, M., Iwata, H., and Sugano, S. (2008). “Development of an Operational Skill-Training Simulator for

Double-Front Work Machine.” Proc., 2008 IEEE/ASME International Conf. on Advanced Intelligent

Mechatronics, 170-175.

Mashiocchi, C., Dark, V., and Parkhurst, D. (2006). “Evaluation of Virtual Reality Snowplow Simulator Training:

Final Report.” Center for Transportation Research and Education (CTRE) Report 06-245, Iowa State Univ.,

Ames, IA.

Panerai, F., Droulez, J., Kelada, J., Kemeny, A., Balligand, E., and Favre, B. (2001). “Speed and safety distance

control in truck driving: Comparison of simulation and real-world environment.” Proc., Driving Simulation

Conf. 2001 (DSC2001).

Schneider, M., and Kiesler, S. (2005). “Calling while driving: Effects of providing remote traffic context.” Proc.,

Conf. on Human Factors in Computing Systems (CHI 2005): Technology, Safety, Community, Portland, OR,

561–569.

Strayer, D., Drews, F., and Burns, S. (2004). “The Development and Evaluation of a High-Fidelity Simulator

Training Program for Snowplow Operators.” Utah Department of Transportation Report UT-04-17, Utah

Dept. of Trans., Salt Lake City, UT.

Tornros, J. (1998). “Driving behavior in a real and a simulated road tunnel: A validation study.” Accident Analysis

and Prevention, 30(4), 497–503.

Wang, X., and Dunston, P. (2005). “Heavy Equipment Operator Training via Virtual Modeling Techniques.” Proc.,

2005 Construction Research Congress, San Diego, CA, 1241-1250.

342

BUILDING INFORMATION MODELLING

BIM Server: Features and Technical Requirements--------------------------------------345

Vishal Singh and Ning Gu

LEED Certification Review in a Virtual Environment-----------------------------------355

Shawn O’Keeffe, Mohd Fairuz Shiratuddin and Desmond Fletcher

Changing Collaboration in Complex Building Projects

through the Use of BIM-------------------------------------------------------------------------363

Saskia Gabriël

The Introduction of Building Information Modelling in Construction Projects:

An IT Innovation Perspective-----------------------------------------------------------------371

Arjen Adriaanse, Geert Dewulf and Hans Voordijk

Creation of a Building Information Modelling Course for

Commercial Construction at Purdue University------------------------------------------383

Shanna Schmelter and Clark Cory


BIM SERVER: FEATURES AND TECHNICAL REQUIREMENTS

Vishal Singh, PhD Candidate

The University of Sydney, Australia;

[email protected]

Ning Gu, Lecturer

The University of Newcastle, Australia;

[email protected]

ABSTRACT: Building Information Modelling (BIM) enables intelligent use and management of building information

embedded in object-oriented CAD models. The widespread use of object-oriented CAD tools such as ArchiCAD,

Revitt and Bentley in practice has generated greater interest in BIM. A number of BIM-compliant applications

ranging from analysis tools, product libraries, model checkers to facilities management applications have been

developed to enhance the BIM capabilities across disciplinary boundaries, generating research interests in BIM

servers as tools to integrate, share and manage the model developed in distributed collaborative teams. This paper

presents the findings from a project aimed at eliciting features and technical requirements for a BIM server. The

paper is based on: (a) a case study conducted with a state-of-art BIM server to identify its technical capabilities and

limitations, and; (b) analysis of technical features of current collaboration platforms (CPs) used in the architecture,

engineering and construction (AEC) industry. Analysis of data reveals that the development of BIM servers should

not be entirely for functional and operational purposes. Based on the findings the technical requirements are

classified into four main categories including both operational and support technical requirements (OTR and STR).

They are (a) BIM model management related requirements; (b) design review related requirements; (c) data

security related requirements; and (d) BIM server set-up implementation and usage assisting requirements.

KEYWORDS: BIM, BIM server, collaboration platform, operational technical requirements, support technical

requirement.

1. INTRODUCTION

Building Information Modelling (BIM) is an advanced approach to object-oriented CAD, which extends the

capability of traditional CAD approach by defining and applying intelligent relationships between building elements

in the model. BIM models include both geometric and non-geometric data, such as, object attributes and

specifications. The inbuilt intelligence allows automated extraction of 2D drawings, documentation and other

building information directly from the BIM model. This inbuilt intelligence also provides constraints that can reduce

modelling errors and prevent technical flaws in the design, based on the rules encoded in the software (Ibrahim et al.

2003, Lee et al. 2006). Most recent CAD packages have adopted the object-oriented approach with certain BIM

capabilities. A number of supporting applications have emerged that can exploit the information embedded in the

BIM model for model integration, design analysis, error checks, facilities management (FM), and so on (Khemlani

2007a). The emergence of multiple applications with the ability to directly use and exchange building information

between them provides opportunities for enhanced collaboration and distributed project development. BIM is

increasingly considered as an IT enabled approach that allows design integrity, virtual prototyping, simulations,

distributed access, retrieval and maintenance of the building data. Hence, the scope of BIM is expanding from the

current intra-disciplinary collaboration through specific BIM applications to multi-disciplinary collaboration through

a BIM server that provide a platform for direct integration, storage and exchange of data from multiple disciplines.

A BIM server is a collaboration platform (CP) that maintains a repository of the building data, and allows native

applications to import and export files from the database for viewing, checking, updating and modifying the data. In

general, a BIM server by itself has limited inbuilt applications. BIM servers are expected to allow exchange of

information between all applications involved in a building project life cycle including design tools, analysis tools,

FM tools, document management systems (DMS), and so on. In principle, BIM servers aim to provide collaboration

capabilities similar to DMS. However, while DMS are meant for collaboration through exchange of 2D drawings

and documents, BIM servers provide a platform for the integration and exchange of 3D model data with embedded

intelligence. This paper presents the findings from a project aimed at eliciting features and technical requirements

345


for a BIM server. Findings reported in this paper are based on: (a) a case study conducted with a state-of-art BIM

server to identify its technical capabilities and limitations, and; (b) an analysis of technical features of current CPs

used in the architecture, engineering and construction (AEC) industry. Analysis of data reveals that the development

of BIM servers should not be entirely for functional and operational purposes. Based on the findings the technical

requirements for a BIM server are classified into four main categories including both operational technical

requirements (OTR) and support technical requirements (STR). They are (a) BIM model management related

requirements; (b) design review related requirements; (c) data security related requirements; and (d) BIM server set-

up implementation and usage assisting requirements. Findings from this research will enhance BIM server research

and development to better facilitate the adoption of the technologies, leading to greater intelligent and automated

collaboration supports in design and construction.

2. BIM ADOPTION

Based on the literature (Johnson and Laepple 2003, Bernstein and Pittman 2004, Holzer 2007, Khemlani 2007b,

Howard and Bjork 2008) and focus group interviews (FGIs) with key industry players, we have identified the main

factors affecting BIM adoption (Gu et al. 2008, 2009). The perception and expectation of BIM against the industry’s

current practice are summarized in terms of the following three main aspects: tools, processes and people.

1. Tools: Expectations of BIM vary across disciplines. For design disciplines, BIM is an extension to CAD. For

non-design disciplines such as contractors, BIM is an advanced DMS that can take-off data from CAD packages

directly. While there are evident overlaps, BIM application vendors seem to be aiming to integrate the two

requirements. The existing BIM applications are not yet mature for either purpose. Users with CAD

backgrounds, such as designers, expect BIM servers to support integrated visualization and navigation

comparable to the native applications they use. Users with DMS backgrounds, such as contractors, expect

visualization and navigation to be the features of BIM servers that are missing in existing DMS solutions.

2. Processes: BIM adoption would require a change in the existing work practice. An integrated model

development needs greater collaboration and communication across disciplines. A different approach to model

development is needed in a collaborative setting where multiple parties contribute to a centralized model.

Standard processes and agreed protocols are required to assign responsibilities and conduct design reviews and

validation. Experience from DBMS (Database Management System) will be useful for data organization and

management, but organizations will need to develop their own data management practices to suit their team

structure and project requirements. Different business models will be required to suit varied industry needs. A

BIM model can be maintained in-house or outsourced to service providers. In the latter case, additional legal

measures and agreements will be required to ensure data security and user confidence.

3. People: New roles and relationships within the project teams are emerging. Dedicated roles, such as BIM model

manager and BIM server manager will be inevitable for large scale projects. Team members need appropriate

training and information in order to be able to contribute and participate in the changing work environment.

In summary, as BIM matures it is likely to integrate the existing CAD packages and DMS into a single product. For

BIM to succeed and be adopted widely in the industry, all the stakeholders have to be informed about the potential

benefits to their disciplines and the project. Earlier research shows that (1) the lack of awareness, (2) the over-focus

on BIM as advancement of CAD packages only, and (3) the relative downplaying of BIM’s document management

capabilities have inhibited the interest of non-design disciplines of the AEC/FM industry in BIM adoption. A user-

centric BIM research has to be more inclusive, since the success of BIM adoption lies in collective participation and

contribution from all the stakeholders in a building project. For this study, the above understanding of BIM adoption

shows that the development of BIM server technologies is not entirely for functional and operational purposes. BIM

servers should not only have the technological capability to support the collaboration requirements of diverse user

groups, but also provide adequate support features to assist the users in assessing, designing and implementing BIM.

3. RESEARCH METHOD

The lack of industry experience in the use of BIM servers means there is limited feedback from industry on technical

requirements for a BIM server (Gu et al. 2009). This study adopts a combined research method involving:

346


• Case study using a state-of-art BIM server: A state-of-art BIM server is tested on a real-world project with

the following research objectives: (1) To test the current functionalities, usability and limitations of the

BIM server as a CP. (2) To use the result of (1) as a benchmark to propose features and technical

requirements for a BIM server that will address the adoption issues as discussed above.

• Analysis of document-based collaboration platforms in the AEC industry: A review of current document-

based CPs and their use in the industry is conducted to identify features and technical requirements that

users may expect in a BIM server. This enables us to extract essential and relevant functions of

collaborative technologies that are currently in use, to be integrated into the ideal BIM server environment.

4. RESEARCH DATA AND ANALYSIS

4.1 Case study using a state-of-art BIM server

For the case study, a renovation project within an Australian landmark building is chosen. A BIM server:

EDMmodelServer™ and relevant BIM applications are used for space renovation and re-functioning of a service

chamber in the building. Therefore, the existing building data, such as the original design drawings, the existing

infrastructures in the service chamber, and its spatial relationships with other surrounding spaces become important

and increase the complexity of the project. These factors were considered and respected in the case study.

The two main tasks of this case study are (1) the construction of different discipline-specific models and (2) the

integration of the models as an integrated BIM model using EDMmodelServer™. Three disciplines involved in the

case study are architecture, hydraulics, and lighting. Applications used for constructing these discipline-specific

models include ArchiCAD for the architectural model and DDS-CAD for the hydraulic and lighting models. Other

applications used within the case study include: Solibri Model Checker, Solibri Model Viewer, DDS-CAD Viewer

and Octaga Modeller (a plug-in for EDMmodelServer™ for 3D model viewing). The case study tests a wide variety

of issues including building data visualization, analysis and collaboration. Specific features that are tested include

(1) object attributes in the discipline-specific models; (2) intelligent relationships within a discipline-specific model

and between different models in the integrated BIM model; (3) data representation, visualization and access

functions; (4) analysis functions that focus on design analysis and model evaluation; (5) project collaboration and

communication functions.

In general, the BIM server has been found useful for design collaboration. The BIM server has well developed

features for data upload, model integration and information extraction such as report generation. However, some

technological and implementation issues are identified during the case study, which are likely to scale up and

develop into a major roadblock in adopting the server for larger projects. These issues are primarily related to:

• Set-up and access to the BIM server: The model server allows defining access rights and permissions based

on participant roles and responsibilities. In the case study few participants were involved. Even in the simplified

scenario, interventions were required midway through the project and personal meetings were organized to

coordinate activities. In a full-scale project, roles and responsibilities not only are likely to increase in

complexity, but also overlap. Thus, it will be useful to methodologically identify role dependencies and

responsibilities, which are critical to the set-up and access to the integrated BIM model.

• Help function and tutorials: Although the participants in the case studies had some training and familiarity

with the BIM server, during the case study some difficulties were reported in tool usage and technical support.

Participants emphasize the need for improved help functions and tutorials. There are suggestions for open-

source training materials where users can learn form each others’ experience.

• User interface (UI): The UI of the BIM server is reported to be confusing. Users that have extended experience

with native disciplinary applications (e.g. CAD tools), find the BIM server interface non-intuitive. Users from

different disciplinary backgrounds may have different usage patterns, requiring different standard interfaces.

Currently, different data and information are shown and limited in one single window. A flexible, user-friendly

UI that allows customization to suit different user profiles is required. 3D visualization of the plug-ins for

viewing the BIM model is good. However, further tests are required for project with larger data sets.

• Data management and modification: In the BIM server, objects and its operations are well defined. However,

during model integration, object duplication or model conflict may occur especially when the same object is

347


created in parallel in different discipline-specific models. Data back-up is critical, especially in the likelihood of

the crash of the BIM server. The upgrades of the BIM server are organized in downloadable modules. If a new

version is available, a message from the application provider will notify the users when they login to the server.

• Extended functionalities for project communication: Some other features and technical requirements

reported as wish-lists in an earlier study (Gu et al. 2009) were reiterate, with most suggestions related to

improving project communication. For example, although the BIM server allows object tagging and change

notifications, if an object is tagged on the BIM server, and the tagged model is downloaded, the tagged message

is not downloaded. The user needs to manually check and update the tagged object separately in the native

disciplinary model. Flexibility to download the appended project communication records along with the model

may be useful. It was also suggested that the BIM server should provide more synchronous communication

features such as an embedded instant messenger for the ease and clarity of project communication.

4.2 Analysis of document-based collaboration platforms in the AEC industry

The analysis includes the examination of existing online CPs, primarily DMS, such as Incite, Aconex, TeamBinder

and Project Centre, along with interviews with industry partners who have adopted the technologies in their

practices. The review was conducted to understand the implementation and application of web-based CPs in the

AEC industry. Such collaborative practices exist within the industry and therefore, they may act as a gauge for using

BIM servers as CPs. The analyses suggests that

• The initial CP set-up is complex. Various dependencies within the process and between activities and people

need to be identified before its operation. The complexity in setting up a BIM server as a CP for building

project development may be even greater because model-based data exchange will require greater coordination

owing to larger size of the data set, varied file formats and tool compatibility issues.

• There are various levels of DMS usage. In some projects, DMS is used across the entire project lifecycle,

involving most of the project participants. In other cases, only some of the project participants coordinate their

activities through DMS, or the DMS is used in specific stages only. This scoping of DMS as a CP is usually

conducted at the initial phases of the project. Such scoping is also critical for BIM server adoption.

• In the current practice, a customized project instruction document is generally developed to serve as a guide for

the project operation. This ensures that as the project develops and the team dynamics change, key

terminologies and standard procedures are agreed and complied by all participants.

• In general, DMS automate the process of uploading, validating, approving and distributing documents. A series

of business rules encoded within the applications at the start of the project automates the decision making such

as which folder to upload documents to, how to validate the document, who to distribute documents to (based

on a distribution matrix), and so on. The inbuilt intelligence in form of business rules and distribution matrix

would require knowledge elicitation from project partners.

• Once the protocols are established, the DMS is configured for the project. The administration team and project

partners are trained, before the system goes online.

• Training programs are available for all functional levels, from the standard tools that all project teams will use,

to the more advanced construction project management tools such as workflows and tender modules. Help is

provided in form of manuals, technical support and video demonstrations.

• The project administrator requires in-depth knowledge of the required document flow process as well as basic

configurations and user requirements on the DMS. In general, a company administrator is appointed to

coordinate with the project administrator and has access rights similar to project administrator.

• Communication is a critical part of all DMS. Most DMS provide multiple modes of communication including

instant messages, SMS texts, emails and voice communications. They support both synchronous and

asynchronous communications, and are used for multiple purposes, such as direct project communication,

documentation, as well as sending notifications, reminders and clarifications.

348


5. TECHNICAL REQUIREMENTS FOR A BIM SERVER

Based on the case study and the analysis above, the authors have broadly grouped the features and technical

requirements for BIM servers as operational technical requirements (OTR) and support technical requirements

(STR). Depending on the actual project, there may be overlaps between OTR and STR.

• OTR refer to the features and technical requirements needed during the usage of the BIM server in direct

support for a building project. OTR can be further divided into the following three categories.

• BIM model management related requirements: These are directly related to the storage, operation and

maintenance of the BIM model that contains all the building information.

• Design review related requirements: These are specifically related to design review activities, including

various functions needed for design visualization and navigation, as well as team communication.

• Data security related requirements: These features and technical requirements are related to network

security and the prevention of unauthorized access into the system.

• STR, such as, help menus and FAQs, have been recognized as an integral part of technological tools (Dicks and

Lind 1995) and are likely to be critical to technology adoption. In project collaboration tools, such as some of

the DMS (e.g. Aconex, Team Binder, Project Centre, and Incite), support features to facilitate the set-up and

implementation of the CPs include assessment matrices, templates, etc. Hence, besides the common support

features such as help menus, tutorials, and FAQs for facilitating the usage of the BIM server, the other

important part of STR for a BIM server includes project decision support features that facilitate and assist the

set-up and implementation of the BIM server for a particular building project.

• BIM server set-up, implementation and usage assisting requirements: These refer to the functions that

are expected to facilitate and assist the set-up, implementation and usage of the BIM server.

Table 1 illustrates the above classifications with some possible overlaps in between. Sections 5.1 to 5.4 elaborate

each of the features and technical requirements that are listed in Table 1.

Table 1 Features and Technical Requirements for a BIM Server

Features and Technical Requirements

(a) BIM model management related requirements

BIM model organization

• Model repository

• Sub-models, and objects with different levels of details

• Public and private model spaces

• Globally Unique Identifier (GUID) for all object data

• Information Delivery Manuals (IDM) based specifications

Model access and usability

• Secured log-in with access rights

• Hierarchical model administration structure

• Download/upload model, and check- in/ check-out/ check-out with lock

• Version lock and archiving

• Model viewing options

• Documentation and reports

User Interface (UI)

• Customizable interface

• Online real-time viewing, printing and markups

• On-click object property check and modification

(b) Design review related requirements

Design visualization and navigation

Team communication and interaction

(c) Data security related requirements

349


• Features supporting confidentiality, integrity, and availability

• System security

• User authentication

• Data security

• Access control

• Encryption

(d) BIM server set-up, implementation and usage assisting requirements

Project decision support

• Project scoping support

• Software tool compatibility matrix

• BIM scoping support

Server administration support

• System configuration manager

• System configuration layout viewer

• System status viewer

Help support and training

Legal and contractual support

5.1 BIM model management related requirements:

5.1.1 BIM model organization

Features and technical requirements for model management and organization should include the following.

• Model repository: A BIM server should provide a centralized data repository for the building project. This data

repository can be linked to other federated data repositories to increase data capacity and efficiency.

• Hierarchical model structure: A BIM model on a server is organized in a hierarchical structure. For example, at

present the model-tree in EDM has the following hierarchy: project > site > building > building storey.

However, users may want a different model structure based on their requirements, e.g. a client may want to

group projects within a site rather than the other way round, i.e. site > project > building > building storey, and

so on. Thus, BIM servers should support the flexibility to customize the model structure.

• Sub-models, and objects with different levels of details: the BIM server should provide the ability to map

objects with different levels of details. For example, if level 1 detail only shows a rectangular volume for a

room, the level 2 detail of same volume may show all the openings, and doors and windows. Users should be

able to navigate and switch across the different views through simple functions and shortcut keys. In order to

support such functionalities, mapping of objects with different levels of details will be required.

• BIM server should support the ability to store and present objects of the model as text-based information in

repositories, and link 3D object-model with model viewer. Choosing object in one window (text-based model

tree or 3D model in model viewer) should highlight the corresponding data in the other window.

• Object and model history, such as ownership and modification records, should be maintained in the repository.

• Object property: The BIM server should provide the ability to overlay additional object properties. For

example, Quality of survey may not be a default object property. If this is the case, in the BIM server, this can

be an overlayed property linked with each object. Technical issues may arise if the data is downloaded and

uploaded again. Additional technical measures will be required to deal with such issues.

• Public and private model spaces: BIM server should allow differentiation of public and private model. Public

model is accessible to all users with access rights. Private model could be model in progress, but not ready to be

shared with others. However, private models may be shared with a select group of users.

• GUID and IDM: Globally Unique Identifier (GUID) allows each object to be uniquely identified, preventing

duplication. Information Delivery Manuals (IDM) details specifications and approaches for connecting the BIM

350


approach with business processes. BIM servers should enable efficient integration of GUID and IDM to deal

with problems encountered in merging different discipline-specific models.

5.1.2 Model access and usability features

Some of the features and technical requirements related to model usability are discussed below.

• Secured log-in with access privileges: it should be possible to import roles and personnel data from

information flow dependency matrix generated in the BIM server set-up phase. See section 5.4 for details.

• Hierarchical model administration structure: BIM server administration deals with management and

allocation of model access rights, data control and security. Typically, hierarchical administrative structures

exist across distributed teams and in large organizations to manage day-to-day local and global issues. Thus,

BIM sever should allow administrative structures that reflect and support existing organizational practices.

• Download / Upload model: Various modes of interaction for model download/upload are possible to include

download/upload buttons as well as drag and drop options. Ability to download/upload data straight to/from an

email account, which is possible in existing DMS when dealing with documents, will be useful.

• Check-in / check-out and version lock: Check-in options should allow addition of new partial model or

merging with existing model. Again, different modes of interaction are possible to include buttons and drag and

drop capabilities. Similarly, check-out options should allow download of complete model or partial model using

different modes of interaction. A check-out with lock feature should be provided to notify other users that the

checked-out data has been locked and deemed not usable. A version lock feature should be provided to lock

version of the model after sign-off, as a form of archiving.

• Model viewing options: BIM server should support the ability to capture and save screen shots, which is a

standard functionality provided across CAD packages. Other features, such as the option to choose the level of

detail for viewing should be available, i.e. sub-sets should be managed such that users can choose the level of

detail for viewing by selecting options on a checklist, e.g. conceptual block model, space layout model, etc.

• Documentation and reports: When downloading a part model from BIM server, there should be options to

generate reports on parametric, linked, and external information for selected objects and the other objects in the

rest of the model. This information can be in form of a checklist, where users can choose to get details of only

those objects they intend to modify, delete or replace. Ideally, a facility to append this information to objects

will be helpful, but that would be useless until the native applications can receive those additional data.

• BIM servers should support the ability to generate and export PDF or other document formats. This capability

also allows direct offloading of ready to use information to DMS, in which case some users may not need to

access the BIM server. They can continue interacting with DMS as they have been doing at present.

• BIM servers should support the ability to integrate information from product libraries. It should be possible to

create a comparison report for alternative product options.

• Features should be provided to validate rules while uploading the files on the BIM server. Users should have

the option to switch validation check on or off.

• Technical provisions are required for data ownership transfer and handover in a BIM server. These

functionalities should account for security measures to deal with such change of hands, log-ins and passwords.

5.1.3 User Interface (UI)

Other than the standard UI features, the BIM server interface should include: (1) Model tree view and 3D viewer

position; (2) support for online real-time viewing, printing and mark-ups; (3) the ability to click on an object, and

check what sub-sets it belongs to, and (4) the ability to click on an object, and switch between the sub-sets it belongs

to, for another sub-set selection. Users should also be able to customize and choose the available UI functionalities.

5.2 Design review related requirements:

The following basic features and requirements are related to distributed design review.

351


• Team communication and interaction: Distributed design reviews may require parallel video conferencing

and similar interaction media. BIM servers should be compatible with such technologies, and integration of

BIM servers with these technologies will be useful for model navigation and viewing. Organizations may want

to maintain a record of the design review interactions in the repository. Thus, BIM servers should provide the

ability to capture real time interaction data from meetings and online reviews. Some of these interaction

platforms, such as instant document/ message exchange window, may be directly hosted on the BIM server.

• Design visualization and navigation: Building projects often result in large data files, which reduce the online

navigation and viewing capabilities. Hence, for effective design review across distributed teams, the 3D Model

viewers supported by BIM servers should have high data compression capabilities while maintaining the image

quality. It will be useful to provide technical features that allow instant, online mark-up, and tagging on a shared

document, model or object being viewed by design reviewers and users.

5.3 Data security related requirements:

Security of data on BIM server should account for confidentiality, integrity and availability of data. These features

and related technical requirements are discussed below.

• Confidentiality: The data stored on the BIM server should be available to authorised users only, and on the

need-to-know basis. This service is crucial to secure sensitive data from malicious intruders.

• Integrity: All BIM data should be only created, modified and deleted by authorised users having an authorised

access, and should be a subject to integrity cross checking.

• Availability: Data and services provided by a BIM server should be available to users when they need them. As

BIM servers have a role of a central data repository serving simultaneously a range of users involved in a

project, availability requirements are of greater consequence than in specialised systems.

• System security: BIM servers should have user authentication facilities to ensure that only authorised users

can access it. The current BIM servers only partially satisfy the security requirements. While they would

typically provide for internal user authentication in form of log-in and password, they fail to provide other

forms of authentication. Moreover, if sufficient password management is not put in place, by the system and

users alike, such authentication procedures may be regarded as inadequate.

• Data security: BIM server should provide effective data access control, with access privileges to individual

pieces of data, including create, delete, read, write and execute. While access control is typically implemented

by the tested BIM server, it does not provide for fine granularity. It would be desirable for BIM servers to adopt

the Role Based Access Control (RBAC) (Sandhu et al. 1996), which is already used by leading Database

Management Systems (DBMS). RBAC allocates access privileges to roles, rather than users, which greatly

simplifies the privilege management task, especially in such a dynamic environment as BIM. The BIM data

should be protected by means of encryption, both when stored on the system and transferred over a network.

This feature is available in major DBMS, it should be adopted by BIM servers.

5.4 BIM set-up, implementation and usage assisting requirements:

5.4.1 Project decision support

The project decision support consists of three main requirements: (1) Project decision support to identify project

dependencies in terms of people, processes and resources, (2) Software tool compatibility matrix for selection of

tools to be used by each collaborating partner contingent on project requirements and technological capabilities and

limitations of dependent collaborators, and (3) BIM scoping support to decide on BIM approaches contingent on

project requirements and technological capabilities of collaborators.

• Project scoping support: Initially workflow process maps should be developed. Various forms would be a

useful way to gather these data from project team members and the client. These should be accessible online

through a web-based interface with secured log-in. Users should be able to download the forms, work offline,

and upload them when they are filled. Users should be able to save partially filled forms and complete and

submit the same in multiple sessions. Once the data is collected, ideally, users should be able to generate

dependency matrices (Yassine and Braha 2003) automatically, with flexibility to develop/ modify these

352


manually. Graphical representation of the dependencies for easy comprehension and viewing, and text-based

search of required dependency is desirable. For example, if the user enters an activity such as design review, the

viewer should show various dependent activities and people, based on what dependency is being sought to

observe. Dependencies can be based on activities, people or resources. Once the dependencies are approved,

there should be a feature to allow setting up automated reminders and notifications. These notifications should

be sent across the medium of communication preferred by the target user, i.e., fax, email, etc.

• Software tool compatibility matrix is required on the BIM server, which is accessible online, and provides

details of tools in terms of compatibility, data formats and interoperability, and capabilities. A directory of tool

experts, and related online tutorials and FAQs can be linked to the matrix. Project partners can coordinate tool

selection by applying the matrix.

• BIM scoping support: In a typical project there are various levels of roles and associations. Some of the

personnel in the project may not have direct access to the BIM server for various reasons. Also, in order to

successfully use the BIM server as a CP it should be possible to receive and upload building information to the

BIM server through media such as email and fax that are preferred by different users. Such technical capability

is supported by document-based CPs, and, hence, similar expectations exist for the BIM server as a CP. These

requirements enhance the scope of BIM usage in a project. Some technical capabilities such as ability to capture

real-time data from site are also important for on-site/ off-site project coordination.

5.4.2 Server administration support

A BIM server integrates with other tools such as CAD, analysis tools, other discipline specific applications, DMS,

etc. This integrated system should be flexible enough to configure differently to suit different project requirements.

• System configuration manager should ensure that the system configuration complies with project

dependencies, allows interactivity between models, documents and appended information. The system

configuration manager should have a customizable UI. It should support rules that regulate the information

provided to be suitable for, and easily adopted to needs of: (1) different users including designer, contractor,

facility manager, client and so on, (2) different building project life cycle ranging from project identification to

bid, start up, design, contract and operate, and (3) different scales of collaborative projects. Synchronous

communication, e.g., chat room, videoconference, and asynchronous communication, e.g., broadcasting and

email, should be supported to improve the project communication.

• System configuration layout viewer is required that graphically shows how different types of data are linked.

Similarly, system status viewer is required for notification of errors, activities update, update on system

performance, and user status, e.g., how many users are logged-in at a given time.

• Data change register is required to maintain the history of the changes made to the data.

The administration support should also facilitate report generation, data back-ups and archives, either manually or

through a pre-set default value (time or size) for automated activation.

5.4.3 Help support and training

Help support and training is critical to the use of the BIM server as a CP. Training support varies with the roles and

responsibilities of the users. Various types of training materials and approaches can be used in conjunction such as:

(1) traditional training and support tools that include help menus, FAQs and helpdesk. (2) open-source training

materials such as technical support blogs that maintain threads of earlier complaints and resolution methods reported

by users and experts. (3) Project-wiki (Kalny 2007) can be created on the server to share project information and

tool usage information. (4) Interactive tutorials such as those already available from various proprietary tools. (5)

An expert directory maintained in the tool-compatibility matrix can provide points of contact for training support.

5.4.4 Legal and contractual support

The model development, reviewing, uploading, downloading, and analysis activities could be quite complex within

an integrated BIM server environment. Specifying ownership, updating liabilities and responsibilities would need

careful consideration. A BIM server use contract agreement is required which should be signed and agreed upon by

the project partners at project initiation. Business rules are needed as a technical feature for model management and

data organization such as archiving, record keeping, backups, and so on. It should be possible to automatically check

353


if these rules conform to IDM specifications, a comprehensive document that details the approach to using and

developing a BIM model. Among other aspects, the legal and contractual support should ensure that the agreements

account for: (1) intellectual property agreements and policies for data exchange, (2) classification of public and

private data, and (3) correspondence protocols.

6. CONCLUSION

This paper presents features and technical requirements for a BIM server. The study shows that the development of

BIM server technologies should not be entirely for functional and operational purposes. BIM servers should not only

have the technological capability to support the collaboration requirements of diverse user groups, but also provide

adequate supporting features to assist the users in assessing, designing and implementing the BIM approach. As a

result, the developed features and technical requirements have been broadly grouped as OTR and STR. Three

categories of OTR: (a) BIM model management related requirements, (b) design review related requirements, (c)

data security related requirements, and one category of STR: (d) BIM server set-up implementation and usage

assisting requirements are subsequently presented and discussed. Findings from this research will enhance BIM

server research and development to better facilitate the adoption of the technologies, leading to greater intelligent

and automated collaboration supports in design and construction.

7. REFERENCES

Bernstein, P.G. anad Pittman, J.H. (2004). “Barriers to the adoption of Building Information Modeling in the

building industry”, Autodesk Building Solutions.

Dicks, R. S., and S. Lind. (1995). “Logistics of integrating online help, documentation, and training: a practical

example.” Proc. of 13th ann. int. conf. on Systems documentation, Savannah, Georgia, US: 34-38.

Gu, N., Singh, V., Taylor, C., London, K. and Brankovic, L. (2008). BIM: Expectations and a reality check,

Proceedings of ICCCBEX11 & INCITE 2008, Tsinghua University Press, China.

Gu, N., Singh, V., Taylor, C., London, K. and Brankovic, L. (2009) “BIM adoption: expectations across

disciplines.” In: Underwood, J and Isikdag (eds), J Handbook of Research on Building Information

Modelling and Construction Informatics: Concepts and Technologies, IGI Publishing (to appear).

Holzer, D. (2007). “Are You Talking To Me? Why BIM Alone Is Not The Answer.” In: Proc. of 4th ICAASA.

Howard, R. and Bjork, B. (2008). “Building information modelling: Experts’ views on standardisation and industry

deployment.” Advanced Engineering Informatics 22: 271–280.

Ibrahim, M., Krawczyk, R. and Schipporiet, G. (2003). “CAD smart objects: potentials and limitations,” ECAADe

21: Digital design: 547-552.

Johnson, R.E. and Laepple, E.S. (2003). “Digital Innovation and Organizational Change in Design Practice.” CRS

Center Working Paper no. 2, CRS Center, Texas A&M University.

Kalny, O. (2007). “Enterprise Wiki: An Emerging Technology to be Considered by the AEC Industry.” AECbytes

Viewpoint 31, March 19, 2007.

Khemlani, L. (2007a). “Supporting Technologies for BIM Exhibited at AIA 2007.” AECbytes, May 24, 2007.

Khemlani, L. (2007b). “Top Criteria for BIM Solutions: AECbytes survey results.” AECbytes October 10, 2007.

Lee, G., Sacks, R. and Eastman, C.M. (2006). “Specifying parametric building object behavior (BOB) for a Building

Information Modeling system.” Automation in Construction 15: 758 – 776.

Sandhu, R.S., Coyne, E.J., Feinstein, H.L. and Youman, C.E. (1996). “Role-based access control models.”

Computer 29 (2):38-47.

Yassine, A. and Braha, D. (2003). “Complex Concurrent Engineering and the Design Structure Matrix Method.”

Concurrent Engineering 11:165-176.

354


LEED CERTIFICATION REVIEW IN A VIRTUAL ENVIRONMENT

Shawn E. O’Keeffe



[email protected]

Mohd Fairuz Shiratuddin, Ph.D.



[email protected]

Desmond Fletcher



[email protected]

ABSTRACT: The excessive use of energy, water, and regional materials are becoming a worldwide concern. This

has led to councils around the world to develop standards and guidelines for review processes to help manage these

global concerns and promote the design and development of sustainable built environment. These guidelines and

standard review processes are used to reduce the depletion of natural resources, and to make the world healthier

and more sustainable for future occupants. The authors propose a new and innovative way of managing and

harvesting sustainable data for the purpose of LEED (Leadership in Energy and Environmental Design)

certification review. Conventionally, changes to the LEED certification review are sometimes made to compensate

for over expenditure of revenue, but these changes could not be analyzed early in the project life cycle. The authors

is developing a 5D BIM model that allows the viewing and reviewing of LEED information in a VE (Virtual

Environment), allows the analysis of cost to be interoperable with the BIM and the LEED certification review

process, and also allows the analysis of how changes made over time affect the total cost of the LEED review.

KEYWORDS: 5D, BIM, LEED, virtual environment.

1. INTRODUCTION

Around the world, people are concern about energy, water, carbon footprint, indoor and outdoor air quality,

harvesting of regional material, and disposal of waste. Buildings around the world affect the livelihood of all living

species through their consumption of energy, resulting to pollution and ozone depletion. Buildings are also

consuming 5 billion gallons of potable water per day (Krygiel et al, 2008). In the USA, buildings consume

approximately 37% of the world energy, and 68% of the world electricity (LEED-NC V2.2, 2007). For LEED

certified green buildings, Capital E (a premier provider of strategic consulting, technology assessment and

deployment, and advisory services to firms and investors in the clean energy industry), has reported an average

energy savings of 30%, average carbon reduction of 35%, a savings of 30-50% on potable water use, a reduction of

land filled waste of 50-97%. In 2003, Capital E developed an average first cost premium of 2% based off 33 LEED

certified buildings in California (Krygiel et al, 2008).

Selecting regional materials and developing buildings that are sustainable is a huge change for the AEC industry.

Sustainable buildings are not something new, but only recently have governments made sustainable development

mandatory. In 1987, the World Commission on Environment and Development stated to the United Nations that

sustainable development must meet the needs of the present without compromising the ability of future generations

to meet their own needs. A good analogy and example of sustainable design is the Native American teepee (Krygiel

et al, 2008). The teepee is designed using regional materials and it does not compromise the future of the land,

deplete the natural material sources, and the people who will inhabit that land. Teepee materials are recycled back

into the native environment without waste. Currently, a typical USA construction project generates 2.5 pounds

(~1.13 kilograms) of solid waste per square foot of floor area. Forty percent of total waste in the USA is caused by

355


construction and demolition, and LEED is helping some project to achieve an 80% waste diversion rate (LEED-NC

V2.2, 2007).

The statistics bring forth a need to provide an efficient way to review sustainability information using Building

Information Modeling (BIM). In this paper, the authors propose the use of Virtual Environment (VE) and 5D-BIM

concepts for LEED certification review. The authors envisioned that this approach would reduce the duration of

certification process, provide certification information that can be achieved in a timely manner, and able to select

different types of material and how they affect cost. By doing so, the AEC industry in general and project

stakeholders in particular, can benefit from better tracking and forecasting of sustainability information, and

construction events early on in projects life cycle. By reviewing specific LEED credits within a VE, a standard

review process working in parallel with the BIM model can be achieved, unlike the traditional paper-based LEED

review method for certification.

2. 5D-BIM PROTOTYPE FOR LEED CERTIFICATION REVIEW IN VE

The 5D-BIM is a construction–time-simulation model of a virtual building that contains cost and other project

related information (Jernigan, 2008). 5D is the linking of the design and construction model (3D), schedule (4D),

and cost (5D). When utilizing a 5D model, users can change any aspect of the three-dimension (3D), with the other

two remaining dimensions (4D and 5D) be automatically updated to accommodate the change (Jernigan, 2008). 4D

and 5D allow AEC professionals and the project owner to utilize simulations to ensure a project plan is feasible and

efficient (Eastman et. al., 2008). The benefits of using a 5D model include: the improvement in communication and

collaboration among stake holders; and better analysis of site logistics, trade coordination on site, and comparisons

of schedule and tracking of construction progress (Eastman et. al., 2008).

A prototype 5D-BIM model is being developed using a suite of software by Vico Software. Sustainability

information for LEED certification purposes is furnished and included in the prototype 5D-BIM model. Besides

being utilized for LEED certification, the sustainability information can be used for GREEN building record such as

construction material usage and cost, duration of which GREEN materials are installed, CSI (Construction

Specifications Institute) Uniformat Methods of Installation, Task Progress of Material Installation, and when the

materials are completely installed (see figure 1). The 5D-BIM model can also be used for Earned Value Analysis

(EVA). The EVA allows the user to simulate “what-if scenarios” and see how cost is affected over time. Inside the

EVA viewer, the 5D cost related information includes Labor, Material, Sub-contract, Equipment and Other, (see

figure 2). Reports of all EVA query can be exported to Microsoft Excel spreadsheet that allows for further

interoperability and data sharing with other software tools. Users can navigate in real-time through the 5D-BIM

model and access the LEED sustainability information.

For LEED certification review performed within a VE to be successful, a LEED-AP1 or designated person by the

Owner must have an early relationship with the AEC professionals involved in the project, specifically the designers

during the early phases of the project life cycle. Currently, to certify a project, a LEED-AP does not have to be hired

or recognized as the party to make submittals to the Green Building Certification Institute (GBCI). A non-accredited

project member can submit to the GBCI if appointed by the Owner to do so. A LEED-AP certifying a project should

have experience and have officially certified a Green Building meeting the standards given and developed by the

USGBC.

1 A LEED-AP is a person who has passed the LEED-AP exam given by the Green Building Certification Institute (GBCI), a third party

organization that handles the accreditation process for the USGBC. A LEED-AP is accredited by the USGBC to streamline and complete the

LEED certification process.

356


FIG. 1: Prototype of the 5D-BIM model

FIG 2: EVA of the prototype 5D-BIM model

A LEED Project can be certified as one of four different levels of certification depending on how many points are

achieved from the GBCI. The four levels of certification are: Certified; 26-32 points, Silver; 33-38 points, Gold; 39-

51 points, and Platinum; 52-69 points. Type of LEED Project certification depends on what category the project falls

under: LEED for New Construction (NC) and Major Renovation, LEED for Neighborhood Development (ND),

LEED for Schools, LEED for Existing Buildings, LEED Core and Shell, and LEED for Existing Buildings

Operation and Maintenance. The prototype for this paper utilizes the LEED – NC Version 2.2. There are over 900

certified buildings and almost 7,000 more registered seeking certification as of June 2007 (LEED-NC V2.2, 2007).

Figure 3 shows a workflow diagram for a successful inclusion and use of LEED information in a 5D-BIM model.

The LEED information can be viewed and reviewed in real-time within a VE. To include the LEED information into

the 5D-BIM model, a new custom recipe must be created for the model's elements (see figure 4). The recipe is then

saved in the standard and project database and will now contain not only the LEED information, but also labor

LEED

Information

Uniformat

Cost

Cost changing over time

357


methods, cost, and sustainable resources. The standard and project databases allow the model element's recipe to be

interoperable among the Vico software suite. The authors used the Vico software suite to produce an interoperable

information workflow for a LEED-ready 5D-BIM model; i.e. between the 3D model, the estimate, the schedule and

the cost analysis (see figure 5). Figure 5 shows the relationship between cost and LEED credits viewable via the

Vico Cost Explorer Software. The LEED-ready 5D-BIM model can then be viewed and reviewed within a VE using

the 5D Presenter software. In the 5D Presenter, users can access and visualize the cost, the LEED information for

certification, the 4D schedule, and the Earned Value Analysis (EVA). Users can also watch the 5D-BIM model

develop over time (see figures 1 & 2).

1) In Constructor recipes are assigned to model elements and the model must be linked to the project database and

standard database. 2) An Estimator project is created and linked to the project database and standard database. 3) The link is verified to ensure connection between Constructor and Estimator.

4) In Estimator, the model data is exported to Cost Explorer. 5) The link is verified to ensure connection between Estimator and Cost Explorer.

6) In Constructor, the WBS (Work Breakdown Structure) is used to create project Task and Task Summary. 7) In Control, a new project is created. 8) In Constructor, the model data is published to Control.

9) In Constructor, the LEED-ready 5D-BIM model is published to a VE in 5D Presenter.

FIG 3: The workflow

The information necessary for the review of LEED certification is provided by the USGBC. The authors have

embedded LEED information into the model element parameters so that the model element parameters contain a

“credit number” that can be referenced to a digital or paper-based LEED summary checklist. The checklist contains

the credit title, intent of credit, requirements, strategy, and submittal documents. Each credit number has a specific

number of points. Using the LEED-ready 5D-BIM model for certification, users are allowed to access the LEED

credits and points in real-time in the VE. The LEED certification process can be done in a VE if the Credit Numbers

and requirements are embedded into the BIM model during the Design Development Phase of the Project Life

Cycle.

Constructor Estimator 1

Estimator Cost Explorer 4

2

3

5

III

.

II

I

Constructor Control 6

7

5

IV

Constructor 5D Presenter 9

8

LEED Recipes

created

358


FIG 4: Development of recipes

FIG 5: LEED cost analysis


The authors believe that it is necessary to innovate and improve the LEED certification process. LEED review

within a VE allows the AEC industry to move forward to a more integrated approach. The review for LEED

certification in a VE will help the AEC manage sustainability information when utilizing a BIM model, and the BIM

model can supplement the certification process. LEED certification in a 5D-BIM model provides various benefits

including the reduction of cost of sustainable design, providing various "what-ifs" scenarios to achieve better

sustainable designs, improving real-time communication of certification intents, and reduction of the time wastage

that occurs when waiting for submittals to be approved for certification. Overall, this could lead to healthier and

more efficient built environments. The authors are still working and improving the LEED certification process in a

VE that will lead to a more automated approach. Current ongoing work includes proposing a sixth dimension (6D)

which is energy (see figure 6).

LEED Credit and Points

LEED Recipe

and Uniformat

cost analysis

bubble

diagram.

359


FIG 6: Samples of energy analyses based on the current research work

The idea of new dimensions for the AEC industry is not new. Lee, et al. (2005) proposed nD modeling as an enabler

for construction improvement. An nD model is an extension of the BIM that incorporates multi-aspects of design

information required at each stage of the overall project life cycle. The authors are proposing a new 6th dimension

for the AEC industry. The proposed new dimension is energy and its purpose is to assess the energy cost, energy

standards, and energy efficient quality of all AEC related projects to provide humans with healthier and more

sustainable built environments. Figure 7 below partially shows how 6D can be directly related to 2D, 3D, 4D, and

5D, and also shows the direct link between LEED certification and 6D. The authors believe 6D is a vital and

sustainable part of nD modeling, sustainable design and construction, and BIM.

FIG 7: Proposed 6D

4. REFERENCES

Eastman, C., Teicholz, P., Sacks, R. and Liston, K. (2008), BIM Handbook: A Guide to Building Information

Modeling for Owners, Managers, Designers, Engineers and Contractors, Publisher: Wiley, ISBN-10:

0470185287, ISBN-13: 978-0470185285.

Jernigan, F. (2008), BIG BIM little bim - the practical approach to building information modeling - Integrated

practice done the right way!, Publisher: 4Site Press; 2nd edition, ISBN-10: 0979569923, ISBN-13: 978-

0979569920.

360


Krygiel, E., Nies, B. and McDowell, S. (2008), Green BIM: Successful Sustainable Design with BIM. Publisher:

Sybex, ISBN-10: 0470239603, ISBN-13: 978-0470239605.

Lee, A and Aouad, G. and Cooper, R and Fu, C and Marshall-Ponting, AJ and Tah, JHM and Wu, S. (2005), 'nD

modelling - a driver or enabler for construction improvement?', RICS Research Paper Series, RICS, London,

5 (6) , pp. 1-16.

LEED-NC V2.2 (2007), USGBC: LEED for New Construction. Retrieved on July 15, 2009 from

http://www.usgbc.org/DisplayPage.aspx?CMSPageID=220.

361


CHANGING COLLABORATION IN COMPLEX BUILDING PROJECTS THROUGH THE USE OF BIM

Saskia Gabriël, PhD,

Henry van de Velde Institute, Artesis University College of Antwerp, Antwerp, Belgium;

[email protected]; http://www.artesis.be

ABSTRACT: Generally, the term Building Information Modeling (BIM) is used in many ways. This paper intends to

give an overview of the current meaning of BIM, as well as to identify the currently experienced advantages and

disadvantages of BIM in the building industry. First, a literature study was performed to investigate which

advantages, disadvantages and barriers to BIM are perceived internationally. Then, personal interviews were done

with building practitioners in Belgium. The results are summed up and compared. It appeared that many

disadvantages can be overcome by time and effort for complex projects. Nevertheless, the implementation of a BIM

still holds disadvantages and difficulties which are to be conquered, especially for smaller projects and smaller

enterprises. The paper concludes with the changing roles of building participants and the need for changing

business processes to optimally collaborate on complex building projects during its entire life cycle.

KEYWORDS: BIM, collaboration, information management, integrated design, interoperability.

1. INTRODUCTION

Over the years, many positive experiences with Building Information Modelling (BIM) and virtual modelling have

been published. In spite of these efforts, building practitioners still seem to experience barriers to the use of BIM-

structures.

The first part of the paper identifies what is intended with the term BIM, since literature defines slightly different

meanings. Due to the multiple use of the term BIM, the advantages and disadvantages of BIM are described under

three different areas which BIM supports: integrated design, interoperable project delivery, and information

management.

Also, it is clearly visible that the role of the different practitioners as well as the collaboration process is changing.

Therefore, the second part of the paper means to hint at how this change is evolving. Understanding evolving roles

can assist in optimising business and process for future building.

A literature study as well as personal interviews with practitioners are performed to answer following questions:

• Which are the advantages and disadvantages of using a BIM?

• What barriers to the use of BIM can be identified?

• How does the use of BIM influence the collaboration process?

The interviews with practitioners were done with architects, engineers, technical advisors, constructors, and others

located in Belgium. Each of them was faced with the same questions. The main goal is to present an overview of

how BIM is seen internationally in the literature and nationally by the practitioners. In brief, the paper can be used to

identify points of attention when using a BIM.

2. BUILDING INFORMATION MODELLING IN USE

2.1 History and present day

In general, a building design is based on the wishes and performance goals as defined by the client. About twenty

years ago, the modelling was done by hand on paper and mostly in 2D. Later on, the design was drawn in CAD

programs on the computer. This CAD-use grew rapidly and transformed into 3D and finally into BIMs.

Today, a Building Information Model (BIM) can be used on several levels. A designer can implement on all the

levels or just a few. Logically, the more levels are implemented, the more benefit the design team can have from the

use of a BIM.

363


The first level contains the virtual representation of materials, constructions and assemblies in a 3D model. This

requires effort from the designer in collaboration with the constructor during early phases of the design process to

implement the acquired properties.

Using BIM on a second level can be described as creativity. Consequently, a designer can easily alter the design and

create complex structures through parametric modelling, without endangering the construction of the objects.

On a third level, a BIM is used to centralise information during all phases of a project. This way, the information is

available to all participants. Access to central information can facilitate collaboration and communication. All can

view, comment on and extract information from the model. When done consistently, conflicts between disciplines

can be solved in the early phases of the design process, leading to lower failure costs in the building process.

A possible fourth and more complex level is the simulation of building parts. For example, energy use, comfort,

daylighting or ventilation strategies can be simulated. Next, the results can be directly implemented in the BIM-

design. Finally, other parts of the building are automatically adjusted and conflicts can be identified quickly.

The fifth and last identified level is computer-driven fabrication of building elements. Computer-aided

manufacturing (CAM) facilitates the creation of complex formed components due to the fact that they are already

digitally detailed. (Kolarevic, 2003)

On the whole, a BIM can be defined as a carrier of all the information supplied by the building disciplines which are

involved during the entire life cycle of a building. The structure can hold data about function, space, materials,

demands, costs and time in a single model, but full implementation remains a challenge for the building industry.

FIG. 1: Integration and use of information in a BIM during the entire life cycle of the building.

2.2 The main reasons

Building projects have become more and more complex due to following requirements (Chen, 2005):

• The limited energy use and operation cost;

• The implementation of innovative techniques;

• Optimisation of the indoor climate and comfort;

• The difficult building sites; and

• The demanded short delivery time for the building.

364


Together, these expectations lay great stress on the participants in a building design process. In order to manage and

fulfil the requirements, a changed process is required for implementation in current projects.

Moreover, the amount of building data has increased tremendously per project but also as the project grows. To keep

an overview and control, the responsible in the design and building process needs to access and control all of the

data in the BIM (see Fig. 2 and Fig. 3).

3. RESEARCH METHODOLOGY

To collect research data, in order to be able to compare with the literature findings, personal interviews were taken

from different parties involved in the building process. More specific, a software developer, a large construction

firm, an architectural firm, a project developer and a facility management firm were interviewed. All of them are

located in Belgium, but have building projects all over Europe. All of these firms have been implementing BIM-

structures for many years.

The query the participants were confronted with, was centralized around three main questions:

• Which are the advantages and disadvantages of using a BIM?

• What barriers to the use of BIM can be identified?

• How does the use of BIM influence the collaboration process?

Other questions were posed to gauge their understanding of BIM, integrated design, operability and information

management. Generally the concepts were understood the same way, although most of the parties had a totally

different understanding of integrated design due to their professional backgrounds.

The main limitation of this study can be identified as a limited group of participants. It would be better to address

more participants from each industry. Also, a more internationally inquiry would help to see the evolutions better in

the field of practice.

4. ADVANTAGES OF BIM-USE

4.1 Literature study

4.1.1 Integrated design

A 3D virtual detailed model assists engineers and architects in visualising their technical solutions for the client.

Changes can easily be made during meetings. As such, the client understands the design more easy. Additionally,

the consultants and engineers can give feedback and advice instantaneously. The impact of demanded changes can

be calculated earlier in the design process. (Howell and Batcheler, 2005)

Because design is transformed into a virtual building, the designer is able to analyse the construction of the building

in detail and anticipate problems early in the design process (Scheer, 2005). Similarly, supporting innovative

solutions in early phases is facilitated. By complete coupling of the 3D-model with 2D-drawings, it becomes

possible to generate attuned schemes and plans with little effort in every phase of the project. (Rundell, 2007,

Schaap and Bouwman, 2006)

Sustainable building is assisted by the use of BIM, due to the transformation of the performance demands into high

technological and sustainable solutions for the client’s demands. The centralised information and early

communication between multidisciplinary participants aids in creating a sustainable building. (Rundell, 2007)

4.1.2 Interoperable project delivery

For some time firms have made use of sophisticated software to assess the building physics in a project. One of the

important set-backs is the limitation of a program which can calculate only few of the variables at once. To analyse

the multiple variables questioned in the design, a BIM and all its data needs to be exchanged with different

simulation and analysis software. The data exchange is facilitated by the Industry Foundation Classes-file (IFC)

format (Zhai, 2006). The information in the model can thus be made interoperable.

365


The export of a BIM towards an IFC format makes the import into most analysis software possible. The greatest

advantage is that the advisor no longer needs to redraw the design to implement their part of the project, which saves

a lot of time and money in the process. Nevertheless, simulation is done only on a part of the building, which is

often simplified to reduce calculation times (Eastman et al, 2008). The information from IFC could therefore be too

complex and requires time to simplify the model.

In general the productivity of design and building teams will improve through sharing and communicating digital

data. Within complex projects, data management is advised to avoid mixing up versions of a model, large quantity

of models being sent around, changing of the wrong data, etc.

Long term relationships between parties in the process can be achieved, so consultants stay involved early in the

process (Howard and Björk, 2008). By working together on several projects, the participants achieve knowledge and

experience from each other. Taking long term collaboration a step further, these partners can be brought together

under a corporate umbrella. This way they can provide more services to the owner, for example operating and

maintaining the building by using the BIM for the entire life cycle and thus managing the total cost of ownership

(Jordani, 2008). Also, work by one member of the project team could benefit another. This implies that benefits

ought to be shared by all involved in the entire process (Howard and Björk, 2008).

FIG. 2: Amount of available information during the design and building process in a traditional building process.

(Kolarevic, 2003)

FIG. 3: Amount of available information during the design and building process by using a BIM.

366


4.1.3 Information management

Decisions are made earlier in an integrated process and this implies time-savings by parallel work (Howard and

Björk, 2008). BIM facilitates the fabrication of building elements with CAM processes since the digital information

is already available and can be extracted from the model indisputable. (Scheer, 2005, Avoort, 2007)

4.2 Personal interviews


Architects indicated the facilitated combination of disciplines as the greatest advantage of applying a BIM in the

design and building process. Moreover, implementation of technical advice in early phases of the design is pointed

out as crucial to avoid extra costs during construction.

Drawing up the model quickly and extracting plans, tables and cost-calculation were defined as the greatest

advantages by the architects, designers, but also by a constructor.


The resistance of clients and designers for integration of sustainable solutions in early design phases has diminished

over the last decade. This implies that more players have evolved towards BIM-use and are now in competition,

which is undoubtedly beneficial for the client.

In all the personal interviews it was corroborated that long-term collaboration works advantageous for the growth of

knowledge and experience of all participants.

Especially the designers added that international collaboration becomes possible with BIM, also improving

knowledge and experience on complex and sustainable buildings.


By managing all data centrally, it is found to be easier and faster to control and recover required information by all

interviewees.

5. DISADVANTAGES OF BIM-USE



Limited creativity, especially in early design phases can be seen as a disadvantage. Designers require large

flexibility in design while maintaining an intelligent BIM. This is a paradox, for intelligence exists of rules for an

element while flexibility exists of freedom to transform the element. (Scheer, 2005)


Each participant in a building project chooses the software that fits his/her work best. This implies the use of several

tools in the design process. Interfaces between tools gain in importance for there is no existing software that meets

all the demands of the project (Hensen and Radošević, 2004, Franken, 2005). Hence, the difficulty of implementing

the system - including software, tools, platforms, networks, etc. - into the business-processes remains a burden on

the tasks and responsibilities of the participants. (Schaap and Bouwman, 2006)

Most benefits of BIM imply long term relationships. In Belgium, there are many ways and contractual forms to

work on a project, but long term relationships are not easily formed. Accordingly, the client requires knowledge of

the possible contracts and decides on the form of collaboration with the help of his advisors. Some forms allow

participants to work together on a series of projects by the use of framework agreements and others support

collaborations throughout the life cycle of the building by including operation and maintenance in the contract.

(Howard and Björk, 2008)

367



The method of export and import of BIM’s is highly anticipated, but still has several disadvantages like: time

consuming preparation, incomplete exports and required customisation, complex and unmanageable files, the

needed change management, little available information about possibilities, etc. (Université de Liège, 2006). Open

standards can be a solution to this problem, but then the requirements for exchange and copyright must be defined in

the contract.

Clear agreements on the way of communication and collaboration are required from the first initiative to avoid

personal gain or interest of the participants (Schaap and Bouwman, 2006). A common goal, shared responsibility

and shared profit are thus important features of attention for the client.

Responsibility of the information in the model is a point of attention. It should be clearly stated who in a project is

responsible for the gathering, delivering, maintaining and controlling the information flow into and from the BIM

(Scheer, 2005). After the delivery of the project the question arises who will update the model during operating and

maintenance phases.



To be able to draw a model quickly, databases are crucial. When not made up before the start of the modelling, a

large amount of time is required to set the elements right. Since a standard interface for extracting tables and cost-

calculation does not exist internationally, each company works with its own, local tools. Much research is being

performed on this subject and it is expected that the local systems are soon to be internationally useable.


Contracts and agreements are made up standard. Integrated design with a BIM requires a total change of thinking

before drawing up the contracts. This is difficult for the client as he/she often has little knowledge nor experience.

Therefore, the architect identifies to be the pre-eminent participant to advise and support this part of the process.


To control the amount of data that is sent around and maintain an overview, a data manager is to be assigned for

complex projects. This person has knowledge on the information flows and controls the data in the BIM. As a result,

this is a completely new function and holds great responsibility, which implies a difficulty in the start of a project.

Many interviewees name the need for such an information manager, but few have assigned one in past projects.

6. BARRIERS TO USING BIM


The learning curve for the BIM-tools can be identified as a clear barrier. The employees working on a project all

need training with the software and the multidisciplinary capabilities. This is a large investment in human resources

for a building project. Therefore, a conversion of the company’s mentality is required towards working with BIM as

an advantage the future. Also, the skills need to be managed and updated, since the functionalities of BIM-tools are

evolving rapidly. (Eastman et al, 2008) Not only the personnel, but also the top of the company needs to be involved

in the process of change towards BIM-use, and need be convinced of the benefits for the business in the future.

(Laverman, 2007)

Other barriers are the limitation and complexity of export and exchange of models and satisfaction to traditional

methods (Yan and Damian, 2008).


Both the designers as the constructors identified the costs of BIM-software too high for a single large project. Unless

the client explicitly demands a BIM, the motivation for BIM-use is quite low. For smaller project the use of BIM

appears not to be profitable. Also, the learning-curve and motivation of individuals in the firm is seen as time and

cost consuming, since the people working with the BIM are required to change their way of working. From

368


traditionally solving problems towards anticipating difficulties in a 3D-model in early design phases is a process that

demands management and time. Changing people demands effort and time. (Laverman, 2007)

Clients apparently do not ask for a BIM which holds all the information needed for the facility management. They

often ask for text documents or calculations, but not for a 3D model of the building. A reason for this could lie in the

incapability of the facility manager to read and interpret the information that lies in a BIM.

The constructors identified in the interviews the importance of involving the management board of the business at

the start of BIM-use. Without the support of the board, it seems almost impossible to integrate the use of BIM for a

project, let alone throughout the entire concern.

In general, constructors (especially the small companies) indicate their satisfaction with traditional methods. They

often have worked in a traditional way for years and feel hesitant to change. Also, they do not work on complex

projects with a large amount of information. On the other hand, many of the larger constructors did feel the need for

change and are implementing the software and the collaboration process on several levels.

7. INFLUENCE OF BIM ON THE COLLABORATION PROCESS

The use of the model and the contained information requires information management, especially in early design

phases, where large quantities of data are produced concurrently. This additional task is best assigned to a party with

the required knowledge and experience.

Communication is clearest when assisted by 3D models, since each participant has his own technical background

which complicates communication. Owing to change from 2D to 3D, investments in software-tools are required.

Also, all participants need to know how to work with the software and understand the importance of working

towards the same goals. They are to be identified as partners in a team, which is frequently experienced as a shift in

collaboration and thinking.

From the personal interviews appeared that not only a changing organisation of the teams for collaboration on one

project, but also long-term commitment for collaboration on multiple projects. The architects noted a change in

thinking along with the technical engineers, more specifically the changing mentality from solving a difficulty in the

design towards avoiding and anticipating conflicts in early design.

Since complex projects require decisions on technical matters, the architect has gained an even more important role

as advisor of the client to help him/her understand the concept and the context. In the same way, other advisors are

necessary to assist the owner in making decisions from early design till the dismantling of the building.

8. CONCLUSIONS

From the personal interviews, it can be seen that the national view on BIM is quite the same as internationally found

in the literature. Often 3D models are used on smaller scale and fewer levels.

Despite the disadvantages and barriers that can be found still, the use of BIM has increased enormously. This clearly

influenced the collaboration process.

Clients of large projects appear to be more and more aware of the advantages for their buildings as well as their

profit in the life cycle of these buildings.

As implementation of a BIM cannot be expected to go over night, the stakeholders in a project need time and some

guidance to adapt their way of thinking as well as their way of working. Both the modelling-software as well as the

collaboration are required to make the process profitable.

Further research into the existing roles of participants in a complex building project is necessary to identify the

evolution towards the collaboration and communication in building projects for the future. Since the designers and

the constructors identified the support of the board for BIM-implementation as highly important, it can be defined

that the implementation of BIM in the entire industry requires a new way of thinking.

Concluding, it can be stated that disadvantages of and resistance to BIM are diminishing, while the experience with

BIM in the building industry is rising. The impacts on energy importance, techniques, and buildings on the whole

cannot go unnoticed.

369


9. REFERENCES

Avoort P.V.d. (2007). Investeren in automatisatie stelt onze toekomst veilig, Schrijnwerk, No. 121, 34-35.

Chen S.-J.G. (2005). An Integrated Methodological Framework for Project Task Coordination and Team

Organization in Concurrent Engineering, Concurrent Engineering: Research and Applications, Vol. 13, No.

3, 185-198.

Eastman C., Eastman C.M., Teicholz P., Sacks R. and Liston K. (2008). BIM Tools and Parametric Modeling, BIM

Handbook: A Guide to Building Information Modeling for Owners, Managers, Designers, Engineers and

Contractors, John Wiley and Sons, Hoboken, USA.

Franken B. (2005). Real as data, Architecture in the digital age: design and manufacturing, (Kolarevic B., editor),

Taylor & Francis, New York & London, 121-138.

Hensen J.L.M. and Radošević M. (2004). Teaching building performance simulation - some quality assurance issues

and experiences, Proceedings of the 21st PLEA international conference Passive and low energy

architecture, Eindhoven, The Netherlands, 1209-1214.

Howard R. and Björk B.-C. (2008). Building information modelling - Experts' views on standardisation and industry

deployment, Advanced Engineering Informatics, Vol. 22, No. 2, 271-280.

Howell I. and Batcheler B. (2005). Building Information Modeling Two Years Later - Huge Potential, Some

Success and Several Limitations, The Laiserin Letter, University of Utah, USA, 1-9.

Jordani D. (2008). BIM: A Healthy Disruption to a Fragmented and Broken Process, Journal of Building

Information Modeling, Vol. 2, 24-26.

Kolarevic B. (2003). Architecture in the digital age: design and manufacturing, Spon Press, New York & London.

Laverman W. (2007). BIM, bir, bom, het Bouwwerk Informatie Model komt er aan!, Building Innovation, March,

40-41.

Rundell R. (2007). 1-2-3 Revit: BIM and Analysis for Sustainable Design, Cadalyst, April 5.

Schaap H.A. and Bouwman J.W. (2006). Toekomst voor het bouwproces; een 3D-benadering, CUR-report 218,

(COINS, editor), Gouda, The Netherlands, 1-39.

Scheer D.R. (2005). Building Information Modeling: What About Architecture?, The Laiserin Letter, University of

Utah, USA, 1-6.

ULg. (2006), System simulation in buildings, Proceedings of the 7th international conference, Les editions de

l’Université de Liège, Belgium.

Yan H. and Damian P. (2008). Benefits and Barriers of Building Information Modelling, Proceedings of the 12th

International Conference on Computing in Civil and Building Engineering, Beijing, China.

Zhai J.Z. (2006). Application of Computational Fluid Dynamics in Building Design: Aspects and Trends, Indoor

and Built Environment, Vol. 15, No. 4, 305-313.

370


THE INTRODUCTION OF BUILDING INFORMATION MODELLING IN CONSTRUCTION PROJECTS: AN IT INNOVATION PERSPECTIVE

Arjen Adriaanse, Dr.

University of Twente/ Ballast Nedam, the Netherlands

[email protected]; www.cme.ctw.utwente.nl/ www.ballast-nedam.nl

Geert Dewulf, Prof.dr.

University of Twente, the Netherlands

[email protected]; www.cme.ctw.utwente.nl

Hans Voordijk, Dr.

University of Twente, the Netherlands

[email protected]; www.cme.ctw.utwente.nl

ABSTRACT: Limited research has been devoted to the introduction and use of BIM across organisational

boundaries in construction projects. The objective of this research is to develop guidelines for a successful

introduction of these BIM technologies. We approach the introduction and use of BIM from an IT innovation

perspective. Because of the explorative nature of the research the introduction and use of BIM will be analysed in-

depth in a construction project. We draw on the principles of action research to apply, validate, and refine our

theoretical concepts. Our research offers two contributions. First, we document and analyse in-depth the

introduction of BIM across organisational boundaries in a construction project. Second, we develop, validate, and

refine guidelines for the introduction of BIM.

KEYWORDS: BIM, construction projects, IT innovation, action research.

1. INTRODUCTION

Building Information Modelling (BIM) can offer many benefits in improving interorganisational communication,

cooperation, and coordination in construction projects. These object-oriented models can be exchanged between

organisations, merged into one model and used, for example, for virtual design reviews, automated quantity take-

offs, and virtual constructability reviews. However, the use and exchange of BIM across organisational boundaries

in construction projects is still limited and not as effective and efficient as it could be.

Numerous scholars have discussed the opportunities and potential benefits of the use of BIM (McKinney and

Fischer, 1998; Whyte et al., 2000; Akinci et al., 2002; Bouchlaghem et al., 2005). In addition, several investigators

documented and analysed the use of these applications in real time construction projects (Bouchlaghem et al., 2005;

Harty, 2005). However, limited research has been devoted to the introduction and use of BIM across organisational

boundaries in construction projects. There is clearly a need for more understanding of these processes so that the

potential benefits of BIM in the future can be realised. The objective of this research is to develop guidelines for the

successful introduction of BIM across organisations in construction projects.

We approach the introduction and use of BIM from an IT innovation perspective (e.g., Cooper and Zmud, 1990;

Swanson, 1994; Swanson and Ramiller, 2004). The IT innovation perspective considers the introduction and use of

BIM as a social instead of a technical phenomenon and enables researchers to analyse its introduction and use in its

social and interorganisational context. It focuses on conditions that need to be secured for successful introduction

and use of BIM. Because of the explorative nature of this research the introduction and use of BIM will be analysed

in-depth in a construction project. We draw on the principles of action research (Baskerville and Wood-Harper,

1996; Baskerville, 1999) and intervene in situations in order to apply, validate, and refine our theoretical concepts.

Our research offers two contributions. First, we document and analyse in-depth the introduction of BIM across

organisational boundaries in a construction project. Second, we develop, validate, and refine guidelines for the

introduction of BIM.

The paper is organised as follows. First, we review the IT innovation processes. Second, the research design is

presented. Third, the results of the case study are summarised. The final part contains conclusions.

371


2. THE IT INNOVATION PROCES

In this paper, we use an IT innovation perspective. Swanson and Ramiller (2004, p.556) define an IT innovation as the

process by which ‘IT comes to be applied in novel ways’. IT innovation may be analysed from the perspective of an

organisation’s subunit, an organisation, or even different organisations (Swanson, 1994). In case of the introduction of

BIM in construction projects to support interorganisational communication, cooperation, and coordination an

interorganisational IT innovation perspective becomes important (Harty, 2005). IT innovation is often viewed as a

process within which various stages may be distinguished: comprehension, adoption, implementation, and assimilation

(Swanson and Ramiller, 2004; see also Cooper and Zmud, 1990). The stages and the considerations and decisions

that need to be made in each of these stages are discussed below from an organisation perspective. In the case study

these theoretical concepts will be translated to the interorganisational context in a construction project and applied to

manage the introduction of BIM.

2.1 Comprehension stage

In the comprehension stage, individuals or other decision-making units actively or passively gather and evaluate

information and scan (1) organisational challenges and opportunities, and (2) IT solutions and its benefits to find a match

between IT solutions and the firm’s own circumstances (Cooper and Zmud, 1990; Rogers, 2003). Through the

sensemaking efforts they learn more about the IT innovation and develop an attitude or stance towards it (Swanson and

Ramiller, 2004). Sensemaking activities can be steered by either organisational needs (pull), technological innovation

(push), or both (Cooper and Zmud, 1990). Based on the sensemaking activities they may decide to become a prospective

adopter.

According to Swanson and Ramiller (2004, p.561) innovators should not take generalised claims about the innovation’s

benefits and applicability at face value but will instead critically examine their local validity. In addition, they should

create situations for rich and context-specific learning to facilitate organisational sensemaking (e.g., demonstrations, site

visits, experimental prototyping) (ibid.).

2.2 Adoption stage

After a prospective adoption decision a deeper consideration of the IT innovation follows in the adoption stage. Both the

business value of the IT innovation and the challenges presented by the prospective change are likely to be weighed

before the organisation decides whether to proceed and commit resources to the innovation (Swanson and Ramiller,

2004). This stage ends with a decision to adopt, not to adopt (i.e., reject) or to defer the adoption of the IT innovation.

The decision for implementing the IT innovation can be based on rational and political negations (Cooper and Zmud,

1990). An adoption decision may be voluntary and based on own considerations. However, an adoption decision may

also be mandated by other firms or by the management of the own firm (Rogers, 2003; Adriaanse, 2007). Rogers (2003)

calls this phenomenon ‘a mandate for adoption’. In these situations there is no other choice than to decide to adopt the IT

innovation.

Swanson and Ramiller (2004, p.562) state that a rational in favour of adopting should be context-specific, rich in its

consideration of local organisational facts, and focussed on the innovation’s potential contribution to the firm’s

distinctive competence. This rationale may point against adoption, or it may favour deferred adoption (ibid.). Sometimes

a firm may decide to experiment or pilot with the IT innovation first before making an adoption decision (Rogers, 2003;

Swanson and Ramiller, 2004). In this view, a firm is implementing parts of the IT innovation before adopting it.

Adriaanse (2007) developed a theoretical model containing categories and subcategories that influence the way actors

use interorganisational IT in construction projects. These categories and subcategories are defined in Table 1. In the

adoption stage, the categories and subcategories can help people responsible for implementing an IT innovation to

identify technical and nontechnical risks and barriers related to the introduction and use of IT in a specific project.

Based on this analysis, they can formulate and implement measures and solutions to control risks and reduce or

eliminate barriers. If barriers cannot be eliminated or reduced enough or if risks cannot be mitigated to acceptable

levels they should not use IT or limit the scope of the use of IT (e.g., limit the scope to only some organisations,

only some aspects of the IT application or only some parts the construction project).

372


Table 1: Categories and subcategories influencing the use of interorganisational IT.

Category, subcategory Definition

1. Personal motivation The extent to which actors are willing to use interorganisational IT themselves. Personal

motivation influences both the willingness of the actors to use IT and their willingness to

invest resources to overcome barriers to the intended use of IT.

1a. Perceived benefits and

disadvantages of IT use

The extent to which actors perceive the use of IT as benefiting and/or disadvantaging them.

1b. Perceived time pressure The extent to which actors perceive that they have to act quickly when using, or considering

the use of, IT. A high level of perceived time pressure can moderate personal motivation

because of the highly perceived benefits of the use of IT.

2. External motivation The extent to which actors are forced by other actors to use IT. External motivation influences

both the use of IT and the efforts made to invest time and money to overcome barriers to the

intended use of IT.

2a. Availability of contractual

arrangements about IT use

The extent to which actors are forced to use IT or other means of communication because this

is mandated in the contract.

2b. Presence of a requesting actor The extent to which another actor requests certain action(s) (e.g. use of IT, or non-use of IT)

to take place and the extent that this request impacts on actors.

3. Knowledge and skills to use IT The extent to which actors know how to use IT. When knowledge and skills are limited, the

actors themselves are the ones restricting the use of IT.

3a. Clarity of procedural agreements The extent to which actors know how to act concerning the IT application (e.g., what

information has to be communicated to whom, and in what form and at what time) and these

actions support the intended use of IT.

3b. Clarity about the operation of IT The extent to which actors know how to operate the application.

4. Acting opportunities The extent to which actors are able to use IT in the intended way. When the acting

opportunities are limited, IT is not able to support the actions of the actors involved.

4a. Alignment between IT and

working practices

The extent to which IT fits in with actors’ working practices in the project and their

organisation(s).

4b. Availability of technical means The extent to which technological aspects restrict actors in using IT in the intended way.

2.3 Implementation stage

Implementation is the critical gateway between the decision to adopt the IT innovation and the routine use of the

innovation within an organisation (Klein and Sorra, 1996). In the implementation stage the IT innovation is developed

and/or tailored to the firm-specific context, installed, and maintained, the organisational procedures are revised and

developed, organisational members are trained both in the new procedures and the IT application (i.e., adaptation)

(Cooper and Zmud, 1990). In addition, organisational members are starting to use the IT application and are induced to

commit to IT application usage (ibid.). Adaptation of the IT innovation may continue based on users’ experience. At the

end of the implementation stage the IT innovation is implemented and users are committed to its use.

Until the implementation stage the IT innovation is a mental exercise of thinking and deciding. Now the IT innovation is

actually put in practice and its use will expand along the way (Rogers, 2003; Swanson and Ramiller, 2004). According to

Swanson and Ramiller (2004, p.562) innovators should be attentive to problems of all kinds, treating them not merely as

obstacles to be overcome but also as potential symptoms of prior misconceptions. They should be sensitive to small

oversights or areas of neglect that might lead to larger failures (ibid.).

373


2.4 Assimilation stage

In the assimilation stage, the IT innovation diffuses across the organisation and becomes routinised in the activities of the

organisation (Purvis et al. 2001). It becomes to be absorbed into the work life of the firm and demonstrates its usefulness

(Swanson and Ramiller, 2004). The use of the IT innovation is seen as a normal activity and the IT innovation is used to

its fullest potential (Cooper and Zmud, 2000). According to Swanson and Ramiller (2004, p.563) innovators should

remain open to surprises, continued learning, and the potential for adaptations that address unanticipated problems or

realise unforeseen potential.

3. RESEARCH DESIGN

The introduction and use of BIM (in this case 3D/4D modelling; 4D = 3D + time) across organisational boundaries will

be analysed from an IT innovation perspective in-depth in a power plant design and construction project. The project is

located in the Netherlands. The civil, structural, and architectural (CSA) work is awarded to the design and construction

firm in December 2007. The amount tendered for the CSA work is about € 170 million, and the project needs to be

finished in 2011. For one part of the project (Pack A: concrete foundations, steel constructions, and cladding of the

power plant) the design and construction firm is only responsible for construction. For another part (Pack B: cooling

water circuit) the design and construction firm is responsible for both design and construction.

As will be described in the next section the design and construction firm introduced 3D/4D modelling in the project. The

engineering company (business unit of the design and construction firm) will make 3D/4D models and will distribute

these to the contractor (also a business unit). They will use 3D/4D models for multidisciplinary design and

constructability reviews and to support communication with the client and subcontractors. One subcontractor is able to

make a 3D model of the steel constructions and will distribute this model to the design and construction firm. The

engineering company merges this model with the other models. Participating organisations are shown in Figure 1. 3D/4D

models are used to support communication with the client but are not distributed to the client.

FIG. 1: Organisations using and exchanging 3D/4D models.

In this research, we draw on the principles of action research (Baskerville and Wood-Harper, 1996; Baskerville, 1999)

and observe events for a period of time, in order to try to get as close as possible to the people being researched. We try

to understand social situations from the standpoint of the participants involved and intervene in situations in order to

apply, validate, and refine our theoretical concepts. More in particular, we try to understand why actors make certain

decisions in the comprehension, adoption, implementation, and assimilation stages and explore how these decisions can

be influenced to manage the introduction of BIM successfully. Therefore, we evaluate the value and usefulness of the

theoretical guidelines.

During the case study, a researcher actively participated in the research and collected data using multiple techniques.

First, the researcher spent most of the time observing participants and informally talking to them. Participant observation

took place during the daily routine and in meetings. The researcher had complete access to all internal project meetings

within the engineering company and the contractor. Second, the researcher conducted many informal and semi-

structured interviews to capture participants’ perceptions and understanding. The researcher tried to see the world from

the participants’ point of view. Finally, the researcher examined documents such as contract documents, minutes of

meetings, and procedures from the quality management system. The researcher followed the project for 7 Months

(December 2007 – June 2008).

374


4. RESEARCH RESULTS

This section presents the introduction process of 3D/4D modelling in the construction project from an IT innovation

perspective. At the end of each stage of the IT innovation process, the findings will be compared with the ‘theoretical’ IT

innovation process as described in Section 2 to show differences, similarities, and refinements. Because of space

limitations, we will only present the highlights of each stage without trying to be exhaustive.

4.1 Comprehension stage

Immediately after the contract was awarded to the contractor in December 2007 one of its work planners contacted the

BIM Program Manager1. He saw some important benefits of 3D/4D modelling in the project. Several weeks ago he

attended an ‘information meeting’ about 3D/4D modelling in which a software vendor showed 3D/4D opportunities and

benefits for construction projects. As a result, in general terms the work planner knew about 3D/4D technologies and its

potential benefits. The BIM Program Manager was able to help the work planner to translate the project challenges to

3D/4D opportunities and its benefits based on experiences in using 3D/4D technologies in two tender projects. Through

these sensemaking activities they could give general claims about 3D/4D opportunities and benefits local validity.

In a follow up meeting they invited a drafter/modeller of the engineering company as well to discuss the 3D/4D

opportunities in the project. The modeller had experiences with 3D/4D modelling and extracting 2D form drawings and

quantities from the 3D model in two tender projects. Based on the discussion they expected that the project would greatly

benefit from 3D/4D modelling (i.e., making 3D and 4D models (including cranes, other critical equipment, and

temporary construction elements) and extracting 2D form drawings and quantities from it). In their view, the main

benefits were:

• Provide better insight into the power plant earlier in the process: 3D models provide better insight to the

contractor’s work planners and works foremen, the client, and subcontractors. This reduces misunderstandings.

• Better insight in the construction process: 4D models provide better insight in who is going to do what at what

moment in time. This insight improves coordination between disciplines and organisations.

• Easier quantity take-off: 3D models can be used for efficient quantity take-offs (e.g., surface areas, volumes).

This is especially interesting for complex parts of the foundations.

• Optimisation of work methods: work planners and works foremen can use 3D/4D models to detect spatial

conflicts in the construction process, show alternatives, and – in the end – optimise work methods.

Together they discussed the risks that are associated with the introduction of 3D/4D modelling in the project and

measures to mitigate these risks as well. These risks and measures are shown in Table 2.

Table 2: Risks and measures to mitigate these risks.

Risks Measures

• A limited number of drafters that is able to

model (i.e., only one experienced

drafter/modeller is available in the

engineering company).

• Limited experience with 3D/4D modelling.

• A limited amount of time to learn 3D/4D

modelling in the project (tight schedule).

• Two drafters will follow a Revit Structure course. One of these drafters can

model in the project together with the experienced drafter/modeller.

• A fallback option to traditional AutoCad will be available.

• Limit the scope first to activities the experienced modeller has done before:

(a) Pack A: 3D/4D models with a limited level of detail (concrete

foundations, buildings), and (b) Pack B: 3D/4D models of the cooling water

circuit, form drawings (detail and reinforcement drawings are excluded),

and quantities.

Together they prepared a presentation for the CEO of the contractor, the project management of the project and some

other members of the project team (contractor and engineering company). Several days later (December 19 2007) they

gave the presentation. They presented the 3D/4D opportunities, the specific benefits for the project, the 3D/4D modelling

activities, the required time investment associated to these, the risks of 3D/4D modelling, and measures to mitigate these

risks. During the presentation there was a lively and enthusiastic discussion between the attendees. Attendees agreed on

1 The BIM Program Manager is responsible for implementing BIM into the design and construction firm.

375


the benefits and saw some additional opportunities. One mentioned the opportunity to extract m2 formwork from the 3D

model. Another mentioned the opportunity to exchange models with subcontractors. Maybe the subcontractors will be

able to provide 3D models. According to the CEO of the contractor 3D/4D activities should focus on optimising work

methods. The models should be used to support communication between, for example, designers, work planners, and

works foremen to optimise the construction process. This was most important to him.

Three persons had to make the decision to implement 3D/4D modelling in the project: the Project Director, the Design

Manager, and the CEO of the contractor. Before the start of the presentation they discussed the introduction of 3D/4D

modelling in the project. They decided not to implement 3D/4D modelling. The Project Director had had some bad

experiences with using these kinds of technologies in another project. The Design Manager was positive about 3D/4D

technology and had followed the introduction and use of 3D/4D modelling in another project (a tender). However, he

was not enthusiastic about 3D/4D modelling in this project because of the lack of a fallback option and the limited

number of experienced modellers. The CEO of the contractor stressed: “We are not going to experiment in this project!

Time pressure is high and stakes are big.”

During the presentation they changed their minds. Especially the examples from another project (a tender), the concrete

benefits for this project, the limited time investment needed, and the controllability of the risks resulted in enough

confidence to give it a try. Because the CEO promised to allocate budget, if needed, for 3D/4D modelling in the project

and the BIM Program Manager was willing to manage the implementation process they decided to implement 3D/4D

modelling in the project. However, first they requested a thorough implementation plan that should be finished 15

January 2009. The BIM Program Manager proposed to write this plan.

The steps taken in the project to successfully complete the comprehension stage are shown in Figure 2. First, the BIM

Program Manager, work planner, and drafter/modeller defined the added value for the specific project based on a ‘3D/4D

information meeting’, their experiences with 3D/4D modelling in tender projects, and their understanding about the

project challenges. Second, they analysed the implementability by assessing the risks of implementing 3D/4D modelling

and measures to mitigate these risks. Third, the BIM Program Manager was willing to manage the 3D/4D adoption (and

implementation) process. Fourth, these three aspects created management support and finally convinced the decision-

makers to make a positive prospective adoption decision. However, a thorough implementation plan (see adoption stage)

needed to be written first before a definite implementation decision would be made. The first step is in line with the

important sensemaking activities as discussed in the ‘theoretical’ comprehension stage in Section 2. However, the other

steps are lacking. These appeared to be essential to create management support and to successfully complete this stage.

FIG. 2: Steps of the comprehension stage.

4.2 Adoption stage

After the ‘go decision’ several meetings took place in which the BIM Program Manager, the work planner, the

experienced drafter/modeller, the Design Manager, and the Head of the Drafting Department (1) worked out the

innovation idea (activities and associated changes in working practices), (2) analysed the implementability of the idea,

376


and (3) discussed parts of the implementation plan. Input for these discussions were the insights, and decisions made in

the comprehension stage. The BIM Program Manager managed this process. The results of the discussions of this

‘adoption group’ are presented below.

4.2.1 Work out the idea: Analysis of activities and changes in working practices

The engineering company will be responsible for 3D/4D modelling. Two drafters/modellers will make the models. The

drafters/modellers have to execute the following general activities: (1) create an object-oriented 3D model, including

large cranes, other critical equipment, and temporary construction elements (3D modelling), (2) create electronic

drawings sheets from the 3D model and add details and annotations to them (drafting; form and detail drawings), (3) link

objects in the 3D model with tasks in the project schedule (4D modelling), (4) extract quantities from the 3D model

(quantity take-off), and (5) merge the models for visualisation. The drafters/modellers will use Revit Structure (3D

modelling, drafting, quantity take-off) and NavisWorks (4D modelling) for executing these activities. The group further

worked out the activities and defined the input and output of these activities. The experienced drafter/modeller is

available for the project. He is experienced in 3D and 4D modelling, extracting form drawings, and doing the quantity

take-off in tender projects. Making detail drawings is a new activity for him. The other drafter/modeller that needs to be

added to the project team is not experienced at all in 3D/4D modelling.

Before the drafters/modellers communicate information formally to the contractor this information needs to be approved

by the Design Manager. Drawings will still be communicated in paper-based form to the contractor. These drawings

instead of the digital model take precedence. The paper drawings will still follow the traditional approval and distribution

processes within the engineering company. The 3D/4D models will not have formal status.

The contractor will use the 3D/4D models for reviews and may decide to use them in meetings with the client and

subcontractors. The project team can use viewers or can ask the drafters/modellers to show aspects in the models. The

use of 3D/4D models and viewers is new to all the contractor’s participants involved.

4.2.2 Analyse the implementability of changes

The BIM Program Manager used the categories and subcategories from Table 1 to analyse the implementability of

the changes for each actor (i.e., experienced and unexperienced drafters, viewer users). He analysed (1) potential

drivers, barriers, and risks, and (2) solutions and measures for these. He refined the risks from the comprehension

stage (see former subsection) based on his increased understanding in the adoption stage about 3D/4D modelling

activities and the associated changes. For both modellers/drafters some results of this analysis are presented in Table

3. The results are grouped to the four theoretical categories.

Table 3: Analysis of implementability of changes for the drafters/modellers.

Analysis (drivers, barriers, risks) Solutions/mitigating measures

Personal motivation

• The experienced drafter/modeller is highly

motivated to use 3D/4D technology and to support

the unexperienced drafter/modeller in learning to

use this technology.

• The unexperienced drafter/modeller is critical

towards some aspects of 3D/4D technology.

• Limit the scope of 3D/4D modelling activities first. Increase the

scope only if the initial scope works well.

• The experienced drafter/modeller will show opportunities and

benefits of 3D/4D technology to the unexperienced drafter.

External motivation

• Project management allows the use of 3D/4D

technology. However, this support will disappear if

the project management becomes disappointed in

the technology or if confidence in the use of the

technology disappears.

• The project team may disturb the drafters/modellers

to show things in the models. This limits their

drafting productivity.

• No other (experienced) drafters/modellers are

available.

• Show the first positive results of 3D/4D modelling as soon as

possible to the project management and team.

• Limit the scope of 3D/4D modelling activities first. Increase the

scope only if the initial scope works well (management support

is needed for that).

• Drafting has priority over other modelling activities.

• Viewer will be available to the project team. They can use the

viewer to analyse 3D/4D models themselves.

• A fallback option to traditional AutoCad needs to be available.

However, drafters/modellers will not be encouraged to use this

377


Analysis (drivers, barriers, risks) Solutions/mitigating measures

option.

• Two drafters will follow a Revit Structure course. One of these

drafters can model in the project.

Knowledge and skills

• The experienced drafter has no experiences with

detail drawings.

• The unexperienced drafter has no experiences with

3D/4D modelling.

• Time pressure is high. It will be difficult to invest

time to learn to use 3D/4D technology.

• The scope of 3D/4D modelling is limited first to limit the

number of new aspects. Increase the scope only if the initial

scope works well.

• User support will be provided by a software vendor (if needed).

• Two drafters will follow a Revit Structure course. One of these

drafters will model in the project.

• The experienced drafter will support the unexperienced drafter

in using Revit Structure.

• A fallback option to traditional AutoCad needs to be available.

However, drafters/modellers will not be encouraged to use this

option.

Acting opportunities

• Unclear version management, approval, and

distribution processes for 3D/4D modelling.

• The Design Manager and BIM Program Manager will develop

new 3D/4D processes and procedures.

4.2.3 Develop the implementation plan

After a while, enough information was available to move from analysis to action. The BIM Program Manager

started to think about the implementation strategy and implementation plan. Because the drafters/modellers had to

start almost immediately (i.e., limited time to prepare), the high time pressure (i.e., limited time to learn), the big

stakes, and to ensure the controllability of the implementation process the BIM Program Manager decided together

with the ‘adoption team’ to choose an incremental (i.e., step-by-step) implementation strategy. This limited the risks

of the introduction and use of 3D/4D modelling and increased the chance to match expectations. There was no

management support for taking risks. The following decisions were being made based on the incremental

implementation strategy:

• Pack A (power plant): the experienced drafter/modeller will start with modelling Pack A (i.e., concrete

foundations and buildings). Because the design and construction firm is only responsible for construction

the modelling activities will be limited to general forms (i.e., a limited level of detail). These modelling

activities will take some days. Steelwork and cladding will be executed by subcontractors. These

subcontractors are not yet selected. If these subcontractors are selected and able to provide 3D models and

if other 3D/4D modelling activities are successful the possibilities of exchanging models with

subcontractors will be further considered.

• Pack B (cooling water circuit): after the unexperienced modeller has followed his training course the

modellers will start with Pack B. The contractor is responsible for both design and construction of the

cooling water circuit. Therefore, the design will be executed ‘in house’ by the engineering company. The

engineering company will start with designing the cooling water intake. The ‘adoption group’ decided to

model the cooling water intake first. The cooling water outfall will only be modelled if modelling of the

cooling water intake goes well and if there is enough capacity to model the outfall. In addition, the

engineering company will start with extracting form drawings from the 3D model. Only if this proves to go

well detail drawings (and maybe reinforcement drawings) will be extracted from the models as well.

Based on these decisions the BIM Program Manager started with writing the project plan. This plan described what

will be done, by whom, when, and how. In the implementation plan the added value of 3D/4D modelling, the 3D/4D

modelling activities (and their input and output), the associated investment, and the main implementation

interventions were described.

January 15 2008 the BIM Program Manager sent the implementation plan for approval to the Project Director. The

Project Director approved the plan. According to the Project Director his 3D/4D knowledge was too limited to

378


assess the plan in a proper way. However, the implementation plan was in line with the given presentation in the

comprehension stage. This was most important to him. However, he would critically follow if 3D/4D modelling

activities match expectations and would terminate the activities if this is not the case. It was most important to him

that drawings will be delivered in time.

The steps taken in the project to successfully complete the adoption stage are shown in Figure 3. First the ‘adoption

team’ worked out the idea (activities, changes). Second, the BIM Program Manager analysed the implementability of the

changes by analysing for each actor (a) drivers, barriers, and risks, and (b) solutions to barriers, and measures to mitigate

risks. The theoretical categories and subcategories were useful. They had been used as a ‘checklist’ to ensure that all

relevant context-specific considerations were made. Third, the BIM Program Manager developed the implementation

plan in which the results of the former steps were incorporated and in which he described concrete activities for the

implementation and assimilation stages. He based his plan on an incremental implementation strategy to limit the risks

of the introduction and use of 3D/4D modelling and to get management support. Finally, this plan created the needed

management support and resulted in an adoption decision by the Project Director. Each of these steps appeared to be

essential to work out the IT innovation, its business value, the challenges presented by its prospective change, and the

implementation plan. These steps together form a context-specific rational in favour of adopting 3D/4D modelling in the

project. In fact these steps are a further refinement of the general aspects of the ‘theoretical’ adoption stage as discussed

in Section 2.

FIG. 3: Steps of the adoption stage.

4.3 Implementation and assimilation stages

After the Project Director approved the implementation plan the BIM Program Manager started with executing

activities as described in the implementation plan. Examples of activities are (a) developing procedures for

approving and distributing 3D/4D models, drawings, and quantities, (b) training and support for the

drafters/modellers, and (c) installation of model viewers. The experienced drafter/modeller gave instruction about

the use of the viewer to the project participants that were interested to use the models.

At the end of January, the experienced drafter/modeller started with making a 3D model of the power plant (Pack

A). He modelled the power plant in several days based on the general 2D drawings of the client. Already during

these modelling activities the modeller found some errors in the client’s drawings. These errors were difficult to

detect in the 2D drawings. The Project Director, engineers, work planners, and works foremen were very

enthusiastic about the 3D model. The 3D model gave a lot of insight in the power plant and its dimensions and

showed design errors. Consequently, more project participants started to use the model viewer to view the 3D model

of Pack A (and later also of Pack B). For example, works foremen used the viewer to gain better insight in the

project, to visualise the construction sequence by adding and removing construction elements, and to extract

quantities. In addition, they used the models in their meetings with subcontractors. After some weeks the

experienced drafter/modeller added cranes to the 3D model because project participants wanted to assess if cranes

could reach parts of the construction site.

379


In February, both modellers started with modelling of the cooling water intake (Pack B). The unexperienced

drafter/modeller just followed a Revit Structure training course. When the unexperienced drafter/modeller started

using Revit Structure the experienced drafter/modeller helped him to model and make form drawings. As a result,

the learning curve for the unexperienced drafter/modeller was steep. However, first the unexperienced

drafter/modeller had to spend more time on drafting in Revit Structure than he used to spend in traditional AutoCad.

According to the unexperienced drafter/modeller he would have shifted to traditional AutoCad drawings if the

experienced drafter/modeller had not supported him. Time pressure was high at that moment. The unexperienced

drafter/modeller was critical towards 3D/4D modelling first. However, after a while when he started to learn to use

Revit Structure he also started to see more benefits. He could work more efficiently in Revit Structure than in

AutoCad. He said: “I don’t want to use AutoCad anymore in my projects.” In addition, because the

drafters/modellers made form drawings in the 3D application forms in the 3D model were kept up-to-date. This was

an important benefit compared to the models of Pack A. The 3D model of Pack A needed to be updated every time

the client sent new drawings.

At the end of February, the contractor started contract negotiations for the steelwork of Pack A with a subcontractor.

This firm appeared to be able to deliver 3D models of steelwork to the contractor. The Project Director supported

the Project Leader of this part of the project to explore the benefits and opportunities of exchanging 3D models with

the subcontractor. The Project Leader and the BIM Program Manager started up the adoption stage for this new 3D

aspect and followed the steps in Figure 3 again. Because of space limitations we are unable to describe this adoption

stage. Here we only want to mention that both the contractor and the subcontractor saw important benefits in

exchanging 3D models between their firms. Together they worked out and tested new 3D working practices. In

addition, the BIM Program Manager analysed the implementability of these new working practices and wrote an

implementation plan. At the end of June he finished the plan and sent it to the Project Director for approval. Note

that another subcontractor (for cladding) was not willing and able to deliver a 3D model. The project management

did not want to mandate the delivery of 3D models towards this firm because this would reduce competition and

raise prices.

In March, the Design Manager decided to make detail drawings of the cooling water intake in the 3D application as

well. In his view, drafting in the 3D application went successfully. However, in April he decided not to expand the

scope of the drafting activities to the cooling water outfall. Main reason was that he expected a huge workload for

coming months. This was caused by many design changes, much detailing of the constructions, and many questions

of the project team to make pictures and show things in the 3D models. It was easier and less risky for the Design

Manager to add a traditional drafter to the drafting team than a Revit Structure drafter/modeller.

In April, the design and construction firm received new drawings of Pack A from the client. Project participants

questioned if the 3D model needed to be updated now. The drafters/modellers were very busy with making drawings

of the cooling water intake. In additions, they could not make a 4D model because the project schedule was not yet

available for Pack A and B. The project management decided not to update the 3D model of Pack A and not to make

a 4D model of Pack A. There was no time available for this update and the added value of the 4D model was limited

at this moment in the project.

It is hard to determine exactly when the assimilation stage started. Theoretically this stage started if 3D/4D

technology was accepted and employed. In the assimilation stage, 3D/4D technology becomes routinised in the

activities of participants and 3D/4D technology is used to its fullest potential. However, the start of this stage differs

between actors and 3D/4D aspects. For example, with form drawings the experienced drafter/modeller moved into

the assimilation stage almost immediately after he started to use Revit Structure. However, with detail drawings this

was completely different. In addition, for the unexperienced drafter/modeller all aspects of 3D/4D modelling were

new. For him, it took some time before he accepted the use of Revit Structure for making form drawings. It even

took some additional months before he could efficiently make form drawings and this became a normal activity. At

the end of June, only making detail drawings in the 3D application were not yet routine for the drafters/modellers. In

addition, they still had to start with 4D modelling.

380


The steps taken in the implementation and assimilation stages are shown in Figure 4. First, the BIM Program Manager

executed the interventions as formulated in the implementation plan and others started to use the applications. Second,

the BIM Program Manager, Project Director, and Design Manager evaluated the interventions and use of 3D/4D

technology. Third, after a while the scope had been partly increased to other aspects (i.e., increase to detail drawings, no

increase to the cooling water outfall). This BIM Program Manager already described possible scope increases in the

implementation plan. However, for another change (i.e., the exchange of 3D models between firms) the adoption stage

had to be started up (see Figure 3). Finally, management support was needed for each change in scope or implementation

plan. These steps are in line with the ‘theoretical’ implementation and assimilation stages as discussed in Section 2. Only

the importance of management support for changes in scope or plans was not mentioned in Section 2. This support was

essential for the project management to control the introduction process.

FIG. 4: Steps of the implementation and comprehension stages.

5. CONCLUSIONS

The objective of this research was to develop guidelines for the successful introduction of BIM across organisations

in a construction project. We approached the introduction and use of BIM from an IT innovation perspective.

Because the insights into these guidelines were limited, an explorative case study approach was conducted in which

we draw on the principles of action research. We developed, validated, and refined IT innovations stages and steps.

The stages and steps increased our understanding of the introduction and use of BIM in a construction project.

This study can be seen as a first step towards developing guidelines for introducing BIM among organisations in

construction projects. The guidelines (i.e., stages and steps) should be further ‘tested’ in other cases. It is suggested

that subsequent research should examine projects in which (a) the use of BIM is mandated in contracts, (b) other

procurement methods are used (e.g., partnering), and (c) other types of IT application are used (e.g., workflow and

document management systems). In addition, future research should be directed towards implementation strategies. In

our case, an incremental implementation strategy was applied successfully to limit the risk of the introduction of BIM.

However, other strategies and their pros and cons should be examined as well to assess their usability in construction

projects.

The results of the study have relevance for practice as well. The stages and steps provide guidance to people

responsible for implementing BIM in construction projects on the question “whether, when, and how to innovate with

BIM in their projects” (Swanson and Ramiller, 2004). Increased understanding of the IT innovation process may help

these people avoiding implementation failures.

6. REFERENCES

Adriaanse, A. M. (2007). The use of interorganisational ICT in construction projects: a critical perspective.

Enschede, University of Twente. Ph.D. thesis.

Akinci, B., M. Fischer, et al. (2002). "Formalization and Automation of Time-Space Conflict Analysis." Journal of

Computing in Civil Engineering, 16(2), 124-134.

Baskerville, R. L. (1999). "Investigating information systems with action research." Communications of the AIS,

2(19), online.

381


Baskerville, R. L. and A. T. Wood-Harper (1996). "A critical perspective on action research as a method for

information systems research." Journal of Information Technology, 11(3), 235-246.

Bouchlaghem, D., H. Shang, et al. (2005). "Visualisation in architecture, engineering and construction (AEC)."

Automation in Construction, 14(3), 287-295.

Cooper, R. B. and R. W. Zmud (1990). "Information technology implementation research: a technological diffusion

approach." Management Science, 36(2), 123-139.

Harty, C. (2005). "Innovation in construction: a sociology of technology approach." Building Research &

Information, 33(6), 512-522.

Klein, K. J. and J. S. Sorra (1996). "The challenge of innovation implementation." Academy of management review,

21(4), 1055-1080.

McKinney, K. and M. Fischer (1998). "Generating, evaluating and visualizing construction schedules with CAD

tools." Automation in Construction, 7(6), 433-447.

Purvis, R. L., V. Sambamurthy, et al. (2001). "The assimilation of knowledge platforms in organizations: an

empirical investigation." Organization Science, 12(2), 117-135.

Rogers, E. M. (2003). Diffusion of Innovations, New York: Free Press.

Swanson, E. B. (1994). "Information systems innovation among organizations." Management Science, 40(9), 1069-

1092.

Swanson, E. B. and N. C. Ramiller (2004). "Innovating mindfully with information technology." MIS Quarterly,

28(4), 553-583.

Whyte, J., N. Bouchlaghem, et al. (2000). "From CAD to virtual reality: modelling approaches, data exchange and

interactive 3D building design tools." Automation in Construction, 10(1), 43-55.

382


CREATION OF A BUILDING INFORMATION MODELING COURSE

FOR COMMERCIAL CONSTRUCTION AT PURDUE UNIVERSITY Shanna Schmelter, Office Engineer of Building Information Modeling

Holder Construction Company

& MS Graduate Alumni of Purdue University

3333 Riverwood Parkway, Suite 400

Atlanta Georgia 30339

[email protected] http://www.holderconstruction.com

Clark Cory, Associate Professor

Purdue University

Department of Computer Graphics Technology

401 North Grant Street, Room 325

West Lafayette, IN 47907-2021

[email protected] http://www.tech.purdue.edu/cgt

ABSTRACT: The focus of this study and paper was developed to determine the need for developing a course at

Purdue University with content relevant to industry trends in: Building Information Modelling (BIM) and

commercial construction practices. The outcome of the study was to develop a course at Purdue University

within the Computer Graphics Technology department that will focus on utilizing BIM for commercial

construction. Students examine 3D geometry, spatial relationships, geographic information, quantities of

materials, and properties of building components in this course. Also, this course will be designed and delivered

to assist students in learning the creation, integration, and utilization of BIM using a computer generated three-

dimensional architectural model within a commercial construction environment. The three-dimensional model,

produced by parametric software is utilized through numerous applications within a real world construction

company. The students acquire these skills by learning how BIM is used in the industry. The students also learn

the processes that make up BIM so they will be able to apply this information in a company where they can use

these techniques or change a company’s ideas to incorporate BIM. It should be due to commercial trends, it is

important that students understand the concepts for future jobs. The purpose of this study was to assess the need

for and receive feedback on the syllabus of the first commercial construction computer graphics course at

Purdue University through a survey that was sent to industry professionals. With the study complete, the course

was implemented and taught during the Spring semester 2009. The paper will review the original study, talk

about the curriculum setup for the course, review the student projects and end with student comments about

course content preparing them for the AEC industry.

KEYWORDS: BIM, Construction Graphic Pedagogy, Visualization,

1. INTRODUCTION

The popularity of Building Information Modelling (BIM) in the commercial construction industry is increasing

everyday (Sullivan, 2007). A recent survey of construction projects and program owners stated that more than

one third of them used BIM on one or more of their projects. This further illustrated that educational settings are

in need of creating new courses and challenging existing ones to facilitate the need of industry (Building Design

and Construction, 2007). The educational and industrial programs that focus on construction graphics are at the

front of this need. Companies are recruiting students with computer graphics skills to BIM positions because of

the modelling knowledge. Most construction companies are slowly redefining their efforts to incorporate BIM

technology and methods. The contractors are “using 3D technology to identify interferences, link data to

schedules, and produce 4D (four dimensional) animations, which help discover dynamic interferences of

construction activities” (Constructech, 2007, p.25). These companies are looking for individuals straight out of

college that are knowledgeable about computer graphics and have a good sense of visualization in construction.

Merriam-Webster (2007) described visualization as the “formation of mental visual images or the process of

interpreting in visual terms or of putting into visible form” (¶ 2). It is suggested that visualization tools may help

enhance the students understanding of certain construction aspects (Messner & Horman, 2003).

383


Computer Graphic Technology (CGT) students at Purdue University who would like to specialize in

Construction Graphic Communication (CGC) are taught residential construction processes with 2D drawings

and 3D tools; however, there many more opportunities in commercial construction that are not explored by the

CGT department that could be explored in similar manners. The New Jersey Institute of Technology (NJIT)

offers a BIM class and design studio where they use Revit Architecture as the main BIM tool to further teach

BIM concepts. They are also researching how to incorporate other BIM tools such as Revit MEP and Revit

Structure into their curriculums (Autodesk, 2007).

Associate Professor of Construction Management, Willem Kymmell at Chico State University in Chico, CA has

created a BIM curriculum that has been incorporated into their school to enable students to understand BIM

concepts and become familiar with BIM tools. BIM assists in information transfer and collaboration settings.

Through this curriculum students are given the opportunity to connect how visualization, communication, and

collaboration all apply to the scope of BIM. Furthermore, the curriculum is unique because it was designed to

suit the attending audience and has basic modules developed to break BIM into manageable concepts. The

students are able to interact with industry projects where they are learning how to document construction

processes and techniques by working with industry professionals. Then the students can compare physical

observations with their BIM observations. The BIM models are still able to be manipulated after this interaction

(Kymmell, 2006).

Currently, there are no commercial construction classes taught at Purdue University through the CGT

department which if taught or introduced could be useful to these students in their future careers. The course

could potentially do several of the following: increase their visualization of 2D into 3D information, increase

their understanding of the coordination of commercial construction documentation and technology, increase

familiarities with the current technologies in BIM, increase understanding of cooperation with architects and

clients, etc. Furthermore, the lack of these visualization tools and this course may be a contributing factor to the

lack of knowledge in commercial construction processes. This course was proposed to greatly enhance the

student’s commercial construction knowledge through the use of BIM techniques to enhance future success in

industry.

This study evaluated the CGT 460: Building Information Modelling for Commercial Construction course

content to provide feedback from the industry who will potentially be hiring Purdue University CGT graduates.

This study collected and analyzed demographic feedback, course feedback, and general course comments to

improve the course and align it with industry standards.

2. WHO NEEDS BIM

BIM is being used in many small companies, industry leading companies, and educational arenas and they are

researching and leading new ways to establish more efficient processes and products. Small to large firms are

now incorporating BIM into their daily activities. Any size firms can have fears of disruptions, cost, and delays

from implementing BIM concepts and software. Small firms have found that transitioning to BIM needs

planning for software and workflow transition, commitment by employees to execute the plan, and measurement

of benefits or losses by using BIM. After implementing the techniques and software, there must be a continual

evaluation of projects using BIM to evaluate the benefits, what types of projects to use it on, and the scope to

use BIM on a project. Using BIM can potentially help smaller firms compete with larger firms. Small firms have

found that even though there are many “culture” changes and technology changes with BIM, they are still more

productive with BIM than without (Kirby, 2007).

Larger projects definitely see the benefits of BIM in multiple disciplines. At the Denver Art Museum, the

Frederic C. Hamilton Building incorporated BIM in the design and coordination which resulted in a safe and

healthy job site. The 3D model and information was used much more than the 2D documents. One reason is

because of the increased availability of BIM tools and technology to larger firms. The Denver Museum model

was originally intended for a specific piece of the structure but then used on many others (Sullivan, 2007).

Educational environments are also using BIM to teach valuable skills to be used in the construction industry.

Northumbria University of the Built Environment in the Newcastle, UK started to offer Architectural

Technology in 1998 to allow students to understand and learn how to correct the gap between design and

construction. Then in 2004 the Architectural Technology program expanded to include BIM practices. The

384


program has affected students positively due to the program interface. It is set up as the students think when

they design a structure which should include the overall scope and 3D. The lecturers found that students could

start to use Revit much quicker than traditional software. Furthermore, this trend allowed the students to learn

more about design principles and sustainability designs than the software (Autodesk, 2007c).

With BIM, “fundamental information needed for the coordination of a project’s design, construction and

operation is captured in digital models at the time design objects are created. The advantages that are offered by

BIM to the building industry provide strong premises to overcome the fragmented nature of the industry. As a

result the industry is likely to see new emerging processes that replace the traditional separation of design,

construction and facilities management” (Salazar et al., 2006, p. 1).

3. SIGNIFICANCE OF THE STUDY

Enhanced visualization is needed to understand many complex structural components that are taught in

commercial construction. There should be a CGT curriculum that teaches commercial construction concepts and

current technology theories such as BIM to enhance student’s visualization of complex commercial structures.

These structures can be difficult to understand and visualize which BIM could assist. Students that are enrolled

in CGT courses that would like to have a future in commercial construction are experiencing frustration due to

the lack of formal commercial construction education. A course addressing this need using BIM would enable

students to have hands-on training with 3D tools where they can use them to further their knowledge in

operations, materials, geometry, spatial relationships, geographic information, quantities, and properties of

commercial building components. These foundations experienced in this new course may benefit CGT students

in their future career development.

The significance of this project was that CGT students will be offered the opportunity to become knowledgeable

in commercial construction and BIM which is becoming an industry standard. This educational opportunity

could potentially improve career possibilities. Similarly, commercial construction aspects of steel and concrete

are currently being taught at the Georgia Institute of Technology using BIM and parametric modelling. Charles

Eastman, a Professor in the Colleges of Architecture and Computing at the Georgia Institute of Technology,

addressed in his research the need for assessment of “a new generation of architectural and building industry

design tools, especially based on parametric design and rule-based design” (BIM Resources @ Georgia Tech,

2007, ¶ 20). As the construction industry and technology progress, education must be at the forefront of these

industry standards to produce the best education possible. This study assumed that by receiving and adopting

feedback from industry professionals, the course would better meet an essential part of their education.

4. PURPOSE

The purpose of the study was to demonstrate BIM techniques to educate students on commercial construction to

assist them in learning “geometry, spatial relationships, geographic information, quantities and properties of

building components” (Purdue University, 2007, p. 1). Students will be able to receive education on professional

practices and “explain how BIM is used in the industry and the processes that make up BIM as they apply to

information in a company that uses BIM techniques” (Purdue University, p. 1). There are no commercial

construction courses in CGT which ultimately can hinder a student’s visualization, understanding of commercial

construction concepts, and future opportunities. This proposed course may increase students’ knowledge,

visualization skills, and perhaps enhance their learning environment. The course objectives are (Purdue

University):

• Increase spatial visualization.

• Increase individual as well as group productivity.

• Coordinate construction documentation and technology.

• Become familiar with the current technologies in BIM and what is utilized for MEP (mechanical,

electrical, and plumbing) systems.

• Understand principles of BIM and incorporate them into current projects in the AEC (architecture,

engineering and construction) industry.

385


• Develop independent and teamwork skills of BIM, and know how to cooperate with architects,

clients, and everyone dealing with the need for information visualization in construction graphics.

• Develop the ability to evaluate and incorporate multiple file formats into one that will evaluate

collision of MEP (mechanical, electrical, and plumbing) and structural systems within a structure.

• Embed and link vital information such as vendors for specific materials, location of details, and

quantities required for estimation and tendering.

• Explain how BIM is used in the industry and the processes that make up BIM as they apply to

information in a company that uses BIM techniques.

• Assess a BIM project and develop a BIM project assessment.

• Become aware of the career opportunities in BIM.

An evaluation tool was utilized to assess the course and laboratory material by professionals in the industry.

This study hypothesized that evaluations from the industry would prove that the course could be effective in

teaching commercial construction and BIM. Also, feedback from the professional sources assisted in adjusting

the course curriculum to be most effective for the CGT students.

5. IMPLIMENTATION OF STUDY INTO CLASSROOM

The normal classroom setup for a Construction Graphic Class at Purdue is two 1 hour lectures in which one is

based on theory and one is a demonstration of technology. The students also have a 2 hour lab toward the end of

the week. This being a 400 level course ended up being tricky to create that paralleled what is done in industry

per professional recommendations. Facility management at Purdue University was called first to obtain a set of

construction documents for one building on campus. After several weeks of negotiation on use of documents

and then scanning of original documents to obtain PDF’s, the student received a complete set of construction

documents for two buildings; Knoy Hall and Electrical Engineering buildings were selected. The construction

documents did not include the MEP or Structural set of prints. Those two items were to come at a later date

which Facility Management did not identify as to a specific date.

The beginning of the semester had started and the students were put into groups of 4 in order to start the model

with the architectural PDF prints. Then the groups were paired up with 2 other groups. The idea was for one

group to work on the architectural prints using Revit Architecture- the second group was to work on the MEP

utilizing Revit MEP and the third group was to work on the structural models utilizing Revit Structural software.

Three weeks into each group’s assignment, they were to switch technologies and models. So if they started

working on the architectural model, they would get the model from MEP or structural and continue to work on

it. This placed some very interesting dilemmas for each group. They had to develop a system of file naming

conventions along with create a standard of software standards in order to pick up where the other group left off.

Software standards and file naming conventions were most important items that the industrial professionals

identified as being extremely important during a project. The industrial professionals get files from multiple

subcontractors and inevitably have to utilize their models or redraw the entire structure from scratch. Either

way, software standards were to become the nightmare for each group. The file naming conventions were

eventually to become a nightmare as well. The instructor of the course made it extremely clear to each group

that the suggestions of the industrial professionals were to be taken very seriously and adhered too.

Each building is a 3 story with multiple basement levels. Each level per building was approximately 25,000

square feet, so it was imperative that the student work together to get each floor completed by the end of the 16

week semester. After each model was completed, the groups were to take them into Navisworks to run a

collision detection. Most industrial professionals are currently utilizing Navisworks in current practices;

therefore it was crucial for students to get introduced to the software. This brought another level of stress into

each student’s life due to the fact that they had to create 3 different models of one building and then run and

then import those modelling into a new software and try to determine how the new software runs and get a

functional report.

The overall schedule for the students to complete the class was 4 weeks for the part one of modelling. Four

weeks for the second part of modelling and 4 weeks for the third part of modelling. The last 3 weeks were to be

for Navisworks component and then a formal presentation in the 16th week identifying modelling through the

semester as well as a Navisworks presentation and lastly a complete summary of group functionality.

386


5.1 Creating the Architectural Model

The student were assigned to a group of 4 and then partnered with 2 other groups for a total of 12 people total.

The first job was to get the Architectural model complete. This was mainly due to the inability of facility

management to get the original structural and MEP construction documents scanned for use in the course. The

groups spent about 4 weeks on the project before the first sign of the other two sets of construction documents

were sent over. During that time, the students were able to get into work sharing and using a central file as

industry does. This allowed all 4 students to work on the same file during the same lab period. This caused a

little bit of a problem in the beginning getting used to all aspects of requesting permission to use certain parts of

the file to work on but once they overcame that lack of knowledge, progress on the architectural model started

moving very quickly. Figure 1 shows one of the architectural models created by the students in the course and

the level of detail produced. The level of detail for the interior of the Knoy Building can be seen in figure 2.

Since the students had extra time to work on the architectural model while the structural and MEP CD’s were

being scanned, some spent the time to include all the exterior limestone detail while others spent time on the

inside of the structure. With the architectural model almost complete the structural and MEP scans finally

arrived. The middle of the semester was just around the corner and the students had a tremendous amount of

work to still accomplish. Both the structural and MEP documents came at the same time. This allowed the class

to work on the project exactly how the industry would. One group would work on the MEP while the other

worked on the structural model.

FIG. 1: Student Work of EE Building (Exterior Detail)

387


5.2 Creating the MEP & Structural Model

The groups were finally introduced to the MEP and the structural construction documents. This is a construction

graphics focus within computer grpahics- so this was the first time most had been able to look at a set of

construction documents specifically for MEP and structural components. Needless to say, there were several

lectures during the week to get them up to speed on how to read them. Each team accomplished the entire

structural model as shown in fig. 3. The MEP was a different story though. While most had been introduced to

MEP in previous courses, none had even drawn or modelled any of the components in the technology. It was an

eye-opening experience for all. The lectures planned were taken up by a Q&A period of MEP review. The

instructor for the course ended up modelling several components for the HVAC as well as the Plumbing in order

for them to see how the components went together in the field. And while the structural model went very

smoothly in class, the students felt the stress to get the MEP portion complete. Figure 4 shows the extent of the

MEP completed for the course. A couple of groups were able to complete the HVAC, electrical components

and Plumbing for one or 2 floors, but was not able to complete the entire building MEP.

FIG. 2: Student Work of Knoy Building (Interior Detail)

FIG. 3: Structural Model of Knoy Building

388


5.3 Collision Detection

Each group had 3 independent models in 3 different software products. The goal was to integrate each into

Navisworks and run a collision detection report. The software standards they had developed in the beginning of

the semester were put to the test. While most groups did a good job defining where the model 0,0,0 coordinate

FIG. 4: MEP Model of Knoy Building

FIG. 5: Navisworks Report (Collision Detection)

389


system was located in each model, there were a couple groups that had to go back and move everything

accordingly. Running the interference report also was a challenge in which most spent several hours in lab

during the evening just to get introduced and have an understanding of the new technology used in industry.

Figure 5 shows the typical collision detection technology report where a duct interfered with a structural

component. Each group soon found out there was more to creating BIM models than just the creation portion.

The big picture of how the model was utilized was finally sinking in and more importantly how their roles in

industry play a big part of the construction process. And while they were only able to run a couple of

comparison reports in Navisworks for a single floor, each recognized the time commitment it would take to do a

complete interference report for the entire structure.

5.4 Final Presentation

While all groups had created the entire for the Architectural, Structural and most of the MEP, they still had to

wrap it all together with a formal presentation. The presentation was to be handled as the AEC industry does

when trying to win the bid or contract for an anticipated client. The Instructor and teaching assistant were the

clients for the course and their presentation had to persuade each to sign the winning contract to the group that

presented the best overall project. Each group was given 30 minutes to introduce, present and wrap up their

project. Most groups went all out and each dressed in similar attire while others decided to take the traditional

college apparel approach. The grade of the presentation was approximately 30% of the overall grade for the

entire course, so it was taken seriously enough with the actual content. A complete review of each group

member and well as each model created was required. - along with the accomplishments & pitfalls of each over

the 15 week period. Since the course was focused on the actual BIM model, none of the groups did a set of

formal construction documents from the model, but focused on each model specifically. Renderings were also

created as a means to present what was accomplished during the semester as shown in figure 6 of an interior and

exterior.

FIG. 6: Exterior & Interior Rendering

390


6. CONCLUSIONS

The overall course defined and created from industrial recommendations of BIM modeling was an overall

success. While there were glitches in the beginning due to lack of information, the overall course went very

smoothly. According to end of year evaluations, each student thought the class was challenging but very helpful

in their education and would benefit them later while applying for jobs in the AEC industry. The overall

schedule for the class was changed multiple times through the semester due to lack of information obtained or

lack of knowledge on students understanding of MEP or structural print reading but in the end each came away

with a greater understanding of their future roles in the AEC industry.

The course will be modified slightly to include a complete set of construction documents presented at the end of

the course. Since it is a construction graphics curriculum, it makes common sense to include a set of working

construction documents to be submitted since each student will be producing those every day in industry. Also,

there will be smaller projects. The 2 buildings selected were extremely large in size to accomplish everything in

one semester. Future building projects will be ½ the size of the building selected for this course. The group sizes

will also be cut in half. This will force each student to take on more responsibility as well as get introduced to

more technology.

Each modification was a direct result of information obtained by industry, the instructor’s observation and

experience in AEC industry, or student comments from semester evaluations. Surprisingly, most students were

very vocal when it came to course input for future course content. While most are wrapped up with finishing

finals or interviewing for jobs, these students were trying to help create a course that would benefit future

students. The instructor was pleased to read so many comments about course improvements. The students that

took the class for the first time all mentioned that they see the benefits industrial input and the comments they

included were a way to give back to Purdue for all it has done for them.

The course created from industrial experience and input is an ongoing process at Purdue University in the

Construction Graphics area of focus. Real world experiences with hands on focus are how the technology

students learn best. The AEC industry is going through the greatest changes it has seen in the last 25 years, the

curriculum at Purdue will remain in constant state of change and update due to technological advancements and

how the BIM is utilized in the AEC industry. With the help of industry, the students are receiving the best

education possible.

7. REFERENCES

Autodesk. (2007). Revit building information modelling: BIM goes to school [white paper, electronic version]

http://students2.autodesk.com/ama/orig/BIM_Goes_To_School.pdf

BIM Resources @ Georgia Tech. (2007). http://dcom.arch.gatech.edu/chuck

Building Design and Construction. (2007). BIM adoption accelerating, owners study finds.

http://www.bdcnetwork.com/article/CA6500734.html?nid=2073

Constructech. (2007). BIM builds its case, Constructech Magazine, Vol. 10 No. 9, 25-28.

Kirby L. (2007). Introducing BIM into a small-firm work environment, Small Project Practitioners Knowledge

Community Journal: The American Institute of Architects, 42.

Kymmell W. (2006). Outline for a BIM curriculum.

http://www7.nationalacademies.org/FFC/willem_kymmell_csu.pdf

Messner J. and Horman M. (2003). Using advanced visualization tools to improve construction education,

Proceedings of the CONVR 2003, Conference on Construction Applications of Virtual Reality.

Purdue University. (2007). CGT 460: Building information modelling for commercial construction ,

http://www2.tech.purdue.edu/cgt/courses/cg460

Salazar G., Mokbel H., Aboulezz M. and Kearney W. (2006). The use of the Building Information Model in

construction logistics and progress tracking in the Worcester Trial Courthouse, Proceedings of the Joint

391


International Conference on Computing and Decision Making in Civil and Building Engineering,

Montreal, Canada, 986 – 995.

Sullivan C. (2007). Integrated BIM and design review for safer, better buildings: How project teams using

collaborative design reduce risk, creating better health and safety in projects. McGraw Hill Construction

Continuing Education. http://construction.com/CE/articles/0706navis-3.asp

392

convr 2009 proceedings

Documents

m1t3 m2t1

maki ujiie

thelocationandtimeof

southern mississippi

improving

dalin oil

karin ailland

mohd fairuz