Feb. 10, 2010 Second Review Meeting

Archaeology & Cultural Heritage Application Working Group

Archaeology & Cultural Heritage Application Working Group

Manolis Vavalis, CERETETHMarios Pitikakis, CERETETH

Chiara Catalano, CNRPatrick Salamin, EPFL

1st year AWG membersAlinari 24 ORE, Italy *Arkaia srl, ItalyBK s.r.l., ItalyCentro Interdipartimentale di Servizi di Archeologia– Università degli Studi di Napoli L’Orientale, ItalyCRS4, ItalyCultural Technologies, Jordan *Cyprus Institute, CyprusEuropean Virtual Engineering Technological Centre (EUVE), SpainFoundation of the Hellenic World, Greece *German Research Center for Artificial Intelligence, GermanyGiunti Labs S.r.l., Italy *Institute of Computer Graphics, TU Graz, AustriaMellaniuM, UKMUSEUM VOLKENKUNDE, The NetherlandsSabanci University / Computer Graphics Laboratory, Turkey *Technische Universität Darmstadt, Interactive Graphics Systems Group, GermanyTELECOM, LIFL (UMR USTL/CNRS 8022), FranceUniversity of Patras – High Performance Information Systems Laboratory, Greece *University of Thessaly, Dept. Of History, Archaeology and Social Anthropology, Greece

• Interested only

in A&CH AWG

• Interested in multiple AWGs

• * Based on existing contacts

2nd year AWG new contacts

Geographic Information Visualization and Analysis, Department of Geography, University of Zurich, SwitzerlandGiusi Grimaudo, TESI Archeologia s.r.l, ItalyMario Santana, Raymond Lemaire International Centre for Conservation, BelgiumMarco Rendina, Centro per la Ricerca e lo Sviluppo di Metodologie e Applicazioni di Archivi Storici, ItalyBreuckmann GmbH, Germany Laura Baratin, University of Urbino, ItalyNadezhda Ilyina, 3DreamTeam, Russia & Czech RepublicRiccardo Berta, ELIOS Lab, University of Genoa, ItalyMaria Economou, Dept. of Cultural Technology & Communication, Univ. of Aegean, Greece Stavroula Zoi, Multimedia-Hypermedia Lab, Athens School of Fine Arts, Greece Livio De Luca, MAP-Gamsau Laboratory, CNRS, FranceCommunications Research Centre (CRC), Canada ** ** Questionnaire submitted

Feb. 5, 2010

AWG members summary

Total AWG members: 30Total 18 AWG members acquired in the 1st year (6 based on existing contacts)Total 12 AWG members acquired in the 2nd year (all new contacts)

2nd year activities

Thematic Workshop (organized by CERETETH and IMATI)Attended VSMM09Workshop on “Serious Game in CH”Approached the Underwater Archaeology community (IMATI)Collaboration with other EU projects:

3D-COFORM – Tools and Expertise for 3D Collection FormationPROBADO - Innovative Digital Library Services for Nontextual Documents

Thematic Workshop - organization

The thematic workshop entitled “3D Knowledge Technologies for Cultural Heritage Applications” was held on 12 September 2009, Vienna, Austria, as a half a day event in conjunction with the 15th International Conference on Virtual Systems and Multimedia - VSMM 2009.An official workshop webpage was developed in the FOCUS K3D portal and the workshop was also advertised at the VSMM09 web pages.Our position statements were circulated to the workshop speakers before the event took place.24 participants attended the workshop + 5 people from FOCUS K3D partners (CERETETH, IMATI and EPFL).Most participants came from academic and research institutions.

Thematic Workshop - program

09:00 – 09:20 Introduction to the workshop : the FOCUS K3D initiative (Michela Spagnuolo, IMATI)09:20 – 9:40 3D-COFORM - Tools and Expertise for 3D Collection Formation (Franco Niccolucci, STARC-Cyprus Institute)09:40 – 10:20 Invited talk: The e-documentation of the Past in 3D: A Challenge and a Risk for the Present and the Future (Marinos Ioannidis, EC & STARC-Cyprus Institute)10:20 – 10:40 Serious Games in Cultural Heritage Field (Riccardo Berta, ELIOS Lab, University of Genoa)10:55 – 11:15 Virtual Humans in Cultural Heritage projects (Patrick Salamin, EPFL)11:15 – 11:35 Integrating 3D digital technologies for CH: issues and perspectives (Andrea D'Andrea, Franco Niccolucci)11:35 – 12:05 A semantic-based framework for managing, searching and retrieving 3D resources (Marios Pitikakis, CERETETH & Chiara Catalano, IMATI-CNR)12:05 – 12:20 Knowledge technologies in Cultural Heritage (position statement) (Manolis Vavalis, CERETETH & Univ. of Thessaly)12:20 – 13:00 Open discussion

Thematic Workshop – position statements

Position statements summary:3D digital representations are often neglected in efforts to create large-scale libraries and there is also a lack in effort by cultural heritage institutions to acquire and use knowledge about 3D content.Whereas text and image digitization and management are rather mature technologies, the technology necessary to fully benefit from 3D knowledge management is not yet within easy reach of most cultural institutions and professionals. A major challenge towards semantic 3D data management is to provide effective content-based and semantic-based organization and searching. Content organization is a fundamental service that first helps the user to navigate and formulate queries. Moreover, it allows an efficient retrieval of the content of interest, which could strongly reduce a large amount of unproductive time.

Thematic Workshop – position statements

Current limitations are mostly related to the lack of specialized tools for 3D knowledge management.Open problems and future directions to promote and exploit knowledge technologies for 3D content in the Cultural Heritage and Archaeology domain could be focused on the following challenges:

(a) Facilitate automatic semantic annotation of 3D digital content; (b) Enhancement of data repositories to exploit and reuse semantic annotations; (c) Intelligent discovery and retrieval of 3D models by improving the efficiency of semantic search engines and their integration with geometric search engines.

Thematic Workshop – outcomes

Initial contact has been made to directly participate and contribute to the European Digital Library (EDL).Had the opportunity to interact with people involved with related fields like serious games, virtual environments and simulations, and understand their perspective.The invited talks and the discussion revealed a number of critical problems related to:

Acquisition and processing of 3D dataTraceability to sources and data provenance must be guaranteed. An ontological approach could be a solution.Not only geometry has to be acquired but also colours, materials and the information related to the object have to be preserved.3D compression has to be addressed.

Thematic Workshop – outcomes

Documentation and annotationThe annotations attached to the objects’ geometry might be lost when the geometry changes due to various reasons. Preserving semantics in the process would be crucial.Adopting a holistic approach to heritage documentation, integrating text, 2D and 3D, requires an ontological approach.There is a management issue and conflict between preservation and provenance.The Semantic (3D-) Web and 3D TV has to be accessed also through other devices, such as mobiles.Watermarking has to be addressed.

Search and retrievalIndexing and 3D Object Retrieval have to be addressed.

Thematic Workshop – outcomes

VisualizationReal time management of the 3D data representation and visualization has to be treated, also considering the profile of the users.Different solutions to visualize documented information gathered during the 3D model creation can be envisaged (whether a model is created from scratch or captured/digitized).

StandardizationThere no consensus in the CH world on metadata standards. It is necessary to demonstrate the effectiveness and viability of “better” solutions.In view of long-term preservation, there are many proprietary data formats by hardware vendors.The evolution in Standards for Metadata has to be combined with the revolution in computer hardware and software.

Thematic Workshop – outcomes

General observationsThe global feeling after the discussion was that the audience was in agreement with our point of view and our position statements.Decision makers are not aware of the importance of 3D for Cultural Heritage. In addition, many of the people involved in the domain are not aware of its needs and 3D is often considered as a “gadget”.The audience was not aware of our work in AIM@SHAPE and the tools we developed to manage 3D semantic shapes (geometric and semantic search engines, SR, TR etc), and they were favourably surprised by their usefulness and effectiveness.

Virtual exhibition scenario - motivation

Written in collaboration with an AWG member (company called MellaniuM)Based on their work in using game engines as a solution for the accurate rendering of archaeological reconstructions and museum exhibits. Until the advent of the latest game engines (e.g. UNREAL v2.5) and the wide acceptance of hardware 3D graphical acceleration video cards it was highly impractical to produce virtual buildings and accessory items with high polygon static meshes and photo-realistic textures and 2D graphics which were not subject to debilitating pixellation on close inspection.

Virtual exhibition scenario - motivation

The key to effective virtual realism, especially for fields like archaeology and cultural heritage, is the creation of an environment so well conceived interpretively that the user becomes emotionally involved in the content of the simulation. Users obviously desire to experience a design that has been created in terms of lighting effects, finishes, surface textures, layout and construction details which will lend itself to a complete suspension of disbelief.

(a) (b)

(c) (d)

Temple of Horus complex (courtesy of MellaniuM) (a) Courtyard near the Entrance to the Hypostyle Hall, (b) Passing through the Hypostyle Hall, (c) Inside the Sanctuary, (d) A chariot outside the Temple of Horus complex

Virtual exhibition scenario - motivation

Museum applications in the future could be used in combination with delivery client (e.g. like the Nortel web.alive). It will be possible in the near future with one URL web link click to enter along with up to 50 others to explore and learn about any ancient site or virtual exhibit.

Geometry and Knowledge Synergy

The benefits of adding semantic information in the different stages of the reconstruction and presentation of an artifact or an entire virtual space are twofold.From the modeler/creator of 3D content perspective, re-creating an ancient artifact in 3D or an ancient site (architectural space) is difficult and time-consuming work. 3D reconstruction is a long design process based on

available data (pictures of the remains, archaeological research drawings and maps, etc.) close collaboration with archaeologists and historians (for example a scanned artifact may be incomplete or damaged, and needs to be reconstructed by applying rules like repeating structures, symmetry or boundary conditions, knowledge of similar artifacts etc.)

Geometry and Knowledge Synergy

In archaeology and cultural heritage, object semantics is typically just as important as the actual geometry. Thus, it is a key requirement to assign thematic information to entire objects and to individual geometric elements. This also makes it possible to select, analyze or edit the geometry and the appearance of objects based on semantic criteria.The importation of high polygon models and rich object textures is a key issue for creating the necessary realism. It must be possible to assign aerial or close-range imagery to the individual geometry elements and ideally, the resulting texture information should also be managed.

Geometry and Knowledge Synergy

Another perspective / area of interest is the organization and presentation of archaeological and cultural heritage content to virtual visitors. Developing educational/training application scenarios and environments which are visually complex and information-rich are a very effective learning tool.The overall experience of a student or virtual a tourist is defined by the virtual reality representation / re-creation. The idea of learning as an active, self-directed, and context-dependent process (e.g. walking around, admiring the ancient buildings, interacting with objects etc.) can greatly contribute to gaining new knowledge.

Geometry and Knowledge Synergy

Embedding semantic information and descriptive metadata related to the original source, age, design and existing knowledge on associated artifacts can be connected effectively to any 3D item in the environment and can contribute to the overall experience. By introducing small unobtrusive icons within the 3D models, which can be approached on the screen, the user will automatically be directed to other sources of information (e.g. web pages, images, movies etc) This type of semantic interactivity is vital to produce an environment that will encompass both a truly informative and a sensory experience resulting in an academically accurate and effective educational space.

Vhuman crowd animation workflow

MotivationFocus on the importance of populating virtual environments with virtual humans and how they can be created and animated.Different steps to create a crowd of animated avatars (virtual humans) in order to animate it with a game engine.The requirements and constraints while working with crowds (avatar behavior, collision avoidance, appearance, movement).

Step 1

Crowd Database:import template (= skeleton + mesh + skinning + textures & segmentationMaps)CrowdEngine: generate walk cycles adapted to the skeleton (1 cycle for each speed)import other needed animation files (idle, facial, etc)

CrowdEngine: generate billboards for each animation posture (including all walk cycles…) test simulation

Step 1

Example billboard for one animation sequence

Step 2

Adding visual Variety1: static accessories (hats, glasses,jewels,…):

modeling accessories meshes (1 or 2 levels of details) export accessories in collada format (.dae)designing accessories variety texture and segmentationMaps (4 per texture) .tga or .ddssetting color variety parameters for all segmentation maps per texture custom .xml3dsMax pos/orient/scale accessories to the template + attach to corresponding joint + specify accessory type export attachment data = custom .txt fileimport accessories & attachment data into Crowd Database

CrowdEngine: generate billboards for all accessories (unless too small to be noticed at far distance)

Step 3

Adding visual Variety2: accessories with specific arm posture (bags, boxes, anything carried in hands…):

modeling & attachment of accessories = repeat 8. Static accessoriesspecify joint rigidity (min/max) or clamp values (min/max) for each arm joint custom .xml file (global for all templates)re-import accessories with joint rigidity&clamp data

Step 3

Step 4

Adding Animation Variety1:adding arm gestures to walking (holding a mobile phone, hands in pocket, etc):

3dsMax specifying IK sources and targets for left and/or right hand export targets in custom .txt .tpo filesCrowdEngine: generate additional walk cycles with new IK constraints (1 cycle per speed for each handIK)CrowdEngine: generate billboards for each new animation

modeling & attachment of accessories if needed (cellPhone,…) = repeat 8. Static accessories

Step 4

Step 5

Adding Animation Variety2: walk diversity & emotional state:

add walking emotional state by specifying angular variations on spine and arms (rigidity & clamp values) custom .xml fileglobal behvior parameters: specifying % of vh for each emotion with walk diversity AND with facial animation custom .xml file

Step 5

Step 6

Crowd Simulation:CrowdEngine: generate navigation graph according to environment’s geometry (various possible parameters)add semantic information to navigation graph (by vertexID or by area)trigger ColorVariety & Accessories & Behavior according to scenario (events, navigation graph’s vertex, areas, semantic info of the environment, etc etc)

Example scenarios: Automatic Identification

The model input is processed for automated annotationA generic ontology is availableThe semantic analyzer combines the results of the annotations and the ontology to input it into the inference module

Semantic Analyzer

Generic ontolog

y

Instances of semantic

information

Automated

annotation

GUI

Instantiated ontology

Inference

Open problems and future directions

A&CH 3D digital content is becoming more and more demanding in terms of

management, preservation, delivery mechanisms…

3D A&CH content often ishard to access and interpretheld in multiple internal systems

with non-standard schemas and descriptions.

The unavoidable manual annotation of explicit semantics is not a practical approach

especially when the number of resources is expected to grow fast.

Indicative Open Problems – Future directions

The technology necessary to fully benefit from 3D knowledge management is not yet within easy reach of most cultural institutions.

further progress in research and consolidation of approaches is required in the area of 3D digital libraries.

Long-term preservation, re-use and access to 3D digital objects are major concerns

the development of standardized metadata and ontologies can partially ensure this.

Semantically enriched descriptions, indexing, and retrieval will allow the deep integration of 3D content into Digital Libraries.

Indicative Open Problems – Future directions

Current limitations are mostly related to the lack of specialized tools for 3D knowledge management and methodologies – as also our questionnaire survey revealed.Ensure access to 3D content through the European Digital Library.Develop solutions for improved re-use of 3D datasets by end-users.Developing easy-to-use authoring tools for 3D knowledge management.

Open Problems – Future directions

There are number of research challenges that should be addressed (as discussed in D1.4.1). These challenges include:

Documenting provenance dataThe need for 3D cultural repositoriesVersion control for 3D modelsDigital rights management of artefactsMark-up language to associate 3D geometry to its annotationWorkflow annotationLong-Term Preservation of 3D Data