3d documentation and semantic aware representation of...

EUROGRAPHICS Workshop on Graphics and Cultural Heritage (2016)C. E. Catalano and L. De Luca (Editors)

3D documentation and semantic aware representation of CulturalHeritage: the INCEPTION project

R. Di Giulio1, F. Maietti1, E. Piaia1

1Department of Architecture, University of Ferrara, Italy

AbstractAs part of 3D integrated survey applied to Cultural Heritage, digital documentation is gradually emerging as effective supportof many different information in addition to the shape, morphology and dimensional data. The implementation of data collectionprocesses and the development of semantically enriched 3D models is an effective way to enhance the dialogue between ICTtechnologies, different Cultural Heritage experts, users and different disciplines, both social and technical. The possibility toachieve interoperable models able to enrich the interdisciplinary knowledge of European cultural identity is one of the mainoutcome of the European Project "INCEPTION - Inclusive Cultural Heritage in Europe through 3D semantic modelling", fundedby EC within the Programme Horizon 2020. The project ranges from the documentation and diagnostic strategies for heritageprotection, management and enhancement, to the 3D acquisition technologies. The development of hardware, software anddigital platforms is aimed at representation and dissemination of cultural heritage through ICT processes and BIM addressesto Cultural Heritage assets, up to the implementation of semantic information to a wider and more extensive use of 3D digitalmodels.

Categories and Subject Descriptors (according to ACM CCS): H.5.1 [Information interfaces and presentation]: Artificial, aug-mented, and virtual realities—I.3.5 [Computer Graphics]: Computational Geometry and Object Modeling—

1. Introduction

One of the main challenge to be faced at European level is to con-tribute to an understanding of Europe’s intellectual basis. In theseefforts, new technologies and digital cultural heritage should playan important role. Innovative digital tools are a great opportunityto understand, access, enhance and preserve cultural assets. Infor-mation and Communication Technologies are constantly evolvingand new digital media are increasingly used for accessing and un-derstanding cultural heritage. The enhancement of digital culturalheritage accessibility is the ability to access cultural contents andresources for as many people as possible by using ICT functionali-ties and applications (web sites, data-bases, digital libraries, virtualapplications, etc.) overcoming cultural, environmental and manage-ment barriers for an easy and spread fruition. Nowadays it is possi-ble to integrate different information in order to access cultural as-sets in many different ways and for many different purposes, thanksto new languages of interactive media aimed at innovative ways ofcommunication of cultural heritage.

Beyond the application of ICT for management, research, diag-nosis, conservation and restoration procedures, education and en-hancement, new technologies allow the communication and dis-semination of cultural assets that become more and more accessiblefor new knowledges and experiences [DG16]; through digital tech-

nologies broad categories of users have access to European tangibleand intangible cultural assets. The availability of databases collect-ing different information allow achieving the widest accessibilityand interoperability at a multidisciplinary level.

New applications allow accessing sites and objects from the her-itage site, or such as in labs, museums, classrooms, or even inpersonal homes or offices. Users have more and more interactivepossibilities to access the knowledge about different sites and ob-jects, to exchange the knowledge between each other, and to enrichthe knowledge with their findings and complementary insights bymeans of interactive platforms and social media. The developmentof data capturing technologies and graphic features has maximizedthe enhancement of digital contents for virtual tour applications,serious games and many different immersive and interactive ex-periences that enable a sustainable access and enjoinment of her-itage sites even by young generations. As part of 3D integratedsurvey applied to Cultural Heritage, digital documentation is grad-ually emerging as effective support of many different informationin addition to the shape, morphology and dimensional data. The in-creasing development of 3D laser scanner technologies allows cre-ating high definition databases based on even more detailed three-dimensional morphometric data. These "digital archives" are an ex-tremely valuable research tool in cultural heritage field: the "geo-

c© 2016 The Author(s)Eurographics Proceedings c© 2016 The Eurographics Association.

DOI: 10.2312/gch.20161406

http://www.eg.org

http://diglib.eg.org

http://dx.doi.org/10.2312/gch.20161406

R. Di Giulio, F. Maietti, E. Piaia / the INCEPTION project

metric memory" is essential for knowledge, protection and sustain-able conservation of Cultural Heritage, although there are still somelimits to the exploitation of 3D models obtained by laser scannersurvey. The growing numbers of un-exploited and "un-interpreted"3D models points out the remarkable need for innovative methodsthat could benefit from the informative value provided by new sys-tems for surveying and representations as well as data managementtools.

Advanced 3D documentation methodologies identifying differ-ent layers of data to be recorded for heritage applications is one ofthe main outcome of the European Project "INCEPTION - Inclu-sive Cultural Heritage in Europe through 3D semantic modelling",funded by EC within the Programme Horizon 2020. The projectproposes the enhancement of efficiency in 3D data capturing pro-cedures and devices, especially as regards their suitability and apti-tude for tangible cultural assets: cultural heritage sites, historical ar-chitectures, archaeological sites characterized by non-conventionalfeatures, location and geometries.

1.1. The INCEPTION project

Starting from innovative methodologies for the 3D survey and mod-elling of Heritage buildings and sites, the INCEPTION projectaims to develop new tools for the interoperability and the inclu-sive sharing of three-dimensional models towards new forms of ac-cess and awareness of the European cultural heritage. The project,started in June 2015, is developed by a Consortium of fourteenpartners from ten European countries led by the Department of Ar-chitecture of the University of Ferrara. Academic partners of theConsortium, in addition to the Department of Architecture of theUniversity of Ferrara, include the University of Ljubljana (Slove-nia), the National Technical University of Athens (Greece), theCyprus University of Technology (Cyprus), the University of Za-greb (Croatia), the research centres Consorzio Futuro in Ricerca(Italy) and CARTIF (Spain). The clustering of small medium enter-prises includes DEMO Consultants BV (The Netherlands), 3L Ar-chitects (Germany), Nemoris (Italy), RDF (Bulgaria), 13BIS Con-sulting (France), Z+F (Germany), Vision and Business Consultants(Greece). The project puts into effect one of the main challengesthat the European Commission has launched by the program Hori-zon 2020 and in particular in the Societal Challenge 6 - Work Pro-gramme 2014-2015: to contribute to a deeper awareness and under-standing of European cultural heritage as inspiration for addressingcontemporary challenges, increasing the knowledge of heritage andits different identities. To this purpose, new technologies and digiti-zation processes play a key role since they allow new and enhancedinterpretations of our collective cultural heritage, while contribut-ing to sustainable economic growth.

Simultaneously to strategies aiming at the optimization of a 3Ddata acquisition protocol able to guide the processes of digitizationof cultural heritage, the project will develop nine case studies. Pi-lot projects started with the recognition of the specific needs andrequirements of each building or site, developing specific three-dimensional modelling operations that will make the digital mod-els used by different categories of inter-disciplinary users, popu-lating the INCEPTION platform. The INCEPTION project aimsat an effective knowledge and dissemination of European cultural

heritage through a Stakeholder Panel. This Panel is an assemblyof European institutions, which will provide a meaningful discus-sion with experts in the field of Cultural Heritage not only scien-tifically but also directing research toward those strategies neededby "end users" and institutions to increase knowledge, enhance-ment and dissemination through digital models in order to promotethe inclusiveness and accessibility of European cultural heritage.The institutions involved in the project "demonstration cases" willcooperate in the development of important applications, tools andmethodological processes of great value and interest. The strategiesof dissemination of digital models will involve both in situ appli-cations for visitors, tourists, scholars and researchers, and remoteapplications that enable the widest possible access and knowledgeof cultural heritage.

2. INCEPTION research avenues

INCEPTION aims to release innovation in 3D modelling of culturalheritage through an inclusive approach for time-dynamic 3D recon-struction of buildings, sites and social environments. The overallinclusive approach comprises an effective enrichment of the Euro-pean identity through understanding of how cultural heritage con-tinuously evolves over long periods of time. This innovation isbeyond the state-of-the-art of rather static 3D reconstruction. 3Dmodels generated through INCEPTION methodologies and toolswill be accessible for all types of user. Semantic enrichment willresult in "intelligent" 3D models for multiple purposes dependingon the needs and the level of knowledge of the end-users (e.g. 3Dsemantic models contain geometric information for 3D visualisa-tion, historical information for narration, information for materialconservation, etc.).

The project follows three main objectives. The first one is cre-ating an inclusive understanding of European cultural identity anddiversity by stimulating and facilitating collaborations across dis-ciplines, technologies and sectors. This objective is addressed byenabling 3D digital reconstructions in an open-standard format forHeritage Building Information Modelling (H-BIM). Such modelswill be easily accessible and reusable by researchers, scientists,experts and creative practitioners working in cultural heritage in-dustries in order to promote collaboration across sectors and fa-cilitate cross-disciplinary researches, dissemination, education andbusiness opportunities. Semantic integration of rich narratives withVirtual and Augmented Reality (VR and AR) will create accu-rate perception and deep understanding for specialists and commonusers. The accessibility and analytical quality of the models is sup-ported by Semantic Web Platform with Semantic Query functionto retrieve rich data, as well as to interface between hardware andsoftware for VR and AR. User apps for mobile devices will be de-velop to access and enrich the H-BIM. These apps are designed forspecialists as well as common users to access the socio-economicsignificance for diffusion, dissemination, knowledge and enhance-ment of Cultural Heritage. Demonstration will be done involvingthe members of Stakeholder Panel.

The development of cost-effective procedures and enhancementsfor on-site 3D survey and reconstruction of cultural heritage build-ings and sites is the second objective. Enhancement of the effi-ciency of 3D data capturing procedure and devices is addressed to


196


the suitability and aptitude for the physical cultural resources andassets. Condition assessment are considered as well.

An open-standard Semantic Web platform for accessing, pro-cessing and sharing interoperable digital models resulting from 3Dsurvey and data capturing will be developed. This objective will beachieved through a standard for managing 3D point clouds data atmultiple scales, linked with Geo-spatial Information Systems. Re-lated actions are: defining an open standard format and semanticontology to generate high-quality, reliable and interoperable mod-els of H-BIM [LDS15]; developing and testing APIs (ApplicationProgramming Interfaces) to allow both definition, implementationand interoperability of various software and data models withoutcompromising quality and functionality; implementing multileveldatabases of 3D semantic models oriented towards various usecases such as understanding, enhancement, promotion, manage-ment and enjoyment of cultural heritage and supporting conserva-tion and restoration works.

"Space" and "Time" are two basic dimensions throughout theINCEPTION project. The overall approach to the survey and mod-elling of heritage spaces and geometric dimensions is the opportu-nity to explore and improve the multi-layered conceptual dimen-sion of European heritage. At the same time, one of the mainchallenges of the project is to develop a set of features allowingthe creation, visualization and analysis of 3D H-BIM models overtime, with emphasis on how the modelled heritage evolves in as-sociation with its build and social environments. The so-called IN-CEPTION "Time-machine" will be developed as open-standard Se-mantic Web Platform. The implementation of the dynamic struc-ture of the INCEPTION Platform through models and reconstruc-tions/simulations related to specific historical periods, allows to"move across time and space" enabling features like time machines.

3. Project concept and approach

Scientific and technological approach followed within the project isfocused on technological development and validation mechanism,ranging from Technology Readiness Level 4 to 7 (improvement andvalidation of technical protocols, selected hardware and software,test models in relevant environments, etc.). Within this scientificand technological framework, the main progresses beyond the stateof the art are related to: braking through the segmentation in col-lecting data, assessment and optimization of 3D data acquisitionprotocol and tools, hardware advancement, tailored algorithms formanaging captured data, conversion of point-cloud data to H-BIM,platform to achieve the widest accessibility to 3D models, improve-ment in the achievement of information integrated in 3D models.

The overall concept and project methodology of INCEPTIONis based on five main Actions. Action 1 is related to the develop-ment of a common framework for catalogue methodology, basi-cally by mapping the stakeholders’ knowledge demands and tar-geted implementations by scholars, technicians, citizens and gov-ernments through the identification of key requirements that con-tribute to meet Europe’s societal objectives related to CulturalHeritage [IMTF∗14]. The implementation of an effective, cross-disciplinary and collaborative work methodology allowed to definethe data collection process, case studies setup, selection and uti-

Figure 1: Scheme of the overall project concept and approachdivided into five main Actions.

lization of system and instruments, implementation of semanticallyenriched models and exploitation in education and business.

Action 2 is related to the integrated 3D data capturing. The IN-CEPTION optimized data acquisition protocol is a set of guidelinesto be followed during the planning and performing of a 3D laserscanner survey of Cultural Heritage, and it is referred to an archi-tectural, archaeological, urban and site scale. It is also referred todata management (scan registration, data verification) data storageand archive. It is both a methodological procedure and an optimizedworkflow specification. Further steps of this action are related to theenhancement of functionalities, capabilities and cost-effectivenessof data capturing technologies and documentation instruments.

Action 3 is focused on the identification of the Cultural Heritagebuildings semantic ontology and data structure for information cat-alogue. Integration of sematic attributes with hierarchically andmutually aggregated 3D digital geometric models is indispensablefor management of heritage information. The aims of this stagesare:

• the creation of a shared semantic field for Cultural Heritage tobe integrated in a wiki-like 3D parametric modelling approach;

• the implementation of guidelines for 3D parametric and semantic


197


modelling in an Heritage-BIM environment, based on Open BIMstandard (IFC, IFD, etc.);

• the development of semantic 3D reconstructions of the heritagesites, integrated with intangible information and social environ-ment, structuring digital representation of sites and artefacts tocreate models more accessible and implementable.

Action 4 is the development of the INCEPTION Semantic WebPlatform. The interoperable Semantic Web H-BIM Platform willallow achieving the widest accessibility and interoperability, theuse of three-dimensional models by researchers from different dis-ciplines and non-expert users, minimizing the difficulties of in-teraction with these kind of data, now accessible only by expertsthrough the use of dozens of different software. The implemen-tation of the dynamic structure of the platform through modelsand reconstructions/simulations related to specific historical peri-ods will prove users with a dashboard for accessing survey data,i.e. cloud of points, and building 3D historical simulations. Thecollection of semantic data associated to the models will enablethe information enrichment by users, sharing knowledge and al-low new interpretation and understanding of European cultural as-sets. An historical database repository will be created in relation toEUROPEANA’s main scopes and objectives, to save and share 3Dmodels with the associated semantic information, documents andall the other possible contributions delivered by external users (i.e.Professors, Experts, Students, End-Users). The database will storethe geographic coordinates to display these models into a specifichistorical map managed with a historical geographic informationsystem (HGIS). Moreover, an historical model search engine willbe implemented. This web application will allow to search withspecific keywords contained in the semantic information [DM12].

Action 5, Deployment and Valorisation, will start by developinga wiki-like feature to enable the sharing and enrichment of the in-formation and interpretation of the models by users; the 3D modelswill be delivered in different Apps for numerous of purposes. The"browsing and query interface" will be defined allowing contribu-tions by researchers and experts that actually do not deal with 3Ddata, and enabling a wide and easy access to the data by citizens,non-expert users and public at large [Val16]. Demonstration activi-ties will start at the beginning of the third year pursuing three maingoals: to gather practical input from the stakeholders; to setup a rel-evant environment (realistic or near-to-operational) for testing andvalidation of the research outcomes; and to disseminate the prelim-inary results through real examples with direct involvement of thestakeholders. Every user will be able to upload 3D models (follow-ing the specific Inception standard), semantic data, documents (i.e.pictures, maps, literature), leave comments or other material neces-sary to improve the scientific knowledge. It will be authorised onlyauthenticated users with a verified email address. The user applica-tions and model deployment with integral narratives will be doneremotely, simultaneously or in different moments, from the her-itage site, such as in labs, museums, classrooms, or even in personalhomes or offices. This will provide the users (i.e. visitors/tourists,scholars/researchers) with an interactive possibility to access theknowledge about the sites and objects, to exchange the knowledgebetween each other, and to enrich the knowledge with their findingsand complementary (or contradicting) insights.

4. Results and future developments

The need of a future re-use of such broad and descriptive sourceof measurement data demands new applications to facilitate infor-mation accessing collected in three-dimensional database withoutcompromising the quality and amount of information captured inthe survey. Databases allow users to understand how each survey-phase was carried out (scans, topographic support, images acquisi-tion, etc.) and thus to obtain the maximum possible amount of mor-phological information; this procedure means to work with com-plex interfaces that are based on the programming languages of thesoftware used to complete the survey itself. Currently, efforts indeveloping the user interface are concentrated on H-BIM environ-ment and semantic enrichment and in providing direct or partiallycontrolled access to the large three-dimensional scale models, alsoby means of immersive navigation. The creation of large digitalspaces properly set up in terms of both form and dimensions, willmake possible to navigate, enter, and extract its qualities and speci-fications (measurements, colours, materials, historical documenta-tion, conservation records). However, the user’s needs and desirefor knowledge might be somewhat stymied by such complicatedinterfaces that could be hard to understand. Current efforts are fo-cused on creating immersive and easy-to-use 3D visualizations thatcan be accessed from a wide range of user. The field of experi-mentation underlying the integrated, interdisciplinary research ef-fort shares many aspects (dimension and complexity of the data)with heritage surveys, and the results obtained so far give us reasonto hope that these optimization processes can be exported. Newsimplified navigation interfaces are also being developed [MS16]for users with lower levels of expertise to facilitate access to andnavigation of the three-dimensional models. It is thus clear thatnew visualization and communication modes for the geometricaland measurement information have to be conceived and developedin step with the development and application of three-dimensionalsurveys.

References[DG16] DI GIULIO R.: Towards sustainable access, enjoyment and un-

derstanding of cultural heritage and historic settings. In Proceedings ofthe International Conference SBE Malta 2016. Europe and the Mediter-ranean: Towards a Sustainable Built Environment (2016), Borg R. P.,Gauci P., Staines C. S., (Eds.), Gutenberg Press. 1

[DM12] DORE C., MURPHY M.: Integration of historic building infor-mation modeling (hbim) and 3d gis for recording and managing culturalheritage sites. In Virtual Systems and Multimedia (VSMM), 2012 18thInternational Conference on (2012), IEEE, pp. 369–376. 4

[IMTF∗14] IOANNIDES M., MAGNENAT-THALMANN N., FINK E.,ZARNIC R., YEN A.-Y., QUAK E.: Digital Heritage: Progress in Cul-tural Heritage. Documentation, Preservation, and Protection5th Inter-national Conference, EuroMed 2014, Limassol, Cyprus, November 3-8,2014, Proceedings, vol. 8740. Springer, 2014. 3

[LDS15] LOGOTHETIS S., DELINASIOU A., STYLIANIDIS E.: Buildinginformation modelling for cultural heritage: A review. ISPRS Annals ofthe Photogrammetry, Remote Sensing and Spatial Information Sciences2, 5 (2015), 177. 3

[MS16] MANURI F., SANNA A.: A survey on applications of augmentedreality. Advances in Computer Science: an International Journal 5, 1(2016), 18–27. 4

[Val16] VALTOLINA S.: A storytelling-driven framework for cultural her-itage dissemination. Data Science and Engineering (2016), 1–10. 4


198

3d documentation and semantic aware representation of...

Documents