d1.3 i-treasures final report final - cordis...d1.3 i-treasures final report i-treasures ict-600676...

Project Title: i-Treasures: Intangible Treasures – Capturing the

Intangible Cultural Heritage and Learning the Rare Know-

How of Living Human Treasures

Contract No: FP7-ICT-2011-9-600676

Instrument: Large Scale Integrated Project (IP)

Thematic Priority: ICT for access to cultural resources

Start of project: 1 February 2013

Duration: 51 months

Deliverable No: D1.3

i-Treasures Final Report

Due date of deliverable:

30 April 2017

Actual submission date:

15 June 2017

Version: Version 1 of D1.3

Main Authors: Filareti Tsalakanidou (CERTH)

Project funded by the European Community under the 7th Framework Programme for Research and Technological Development.

D1.3 i-Treasures Final Report i-Treasures ICT-600676

Filename: D1.3_i-Treasures_Final_Report_final.doc of 98

Deliverable title i-Treasures Final Report

Deliverable number D1.3

Deliverable version Version 1

Previous version(s) 0

Contractual date of delivery 30 April 2017

Actual date of delivery 15 June 2017

Deliverable filename D1.3_i-Treasures_Final_Report_final.doc

Nature of deliverable R = Report

Dissemination level PU = Public

Number of pages 98

Workpackage WP1

Partner responsible CERTH

Author(s)

Filareti Tsalakanidou (CERTH), Nikos Grammalidis (CERTH), Kosmas Dimitropoulos (CERTH), Dimitrios Konstantinidis (CERTH), Pierre Roussel (UPMC), Lise Crevier-Buchman (CNRS), Sohaib Laraba (UMONS), Stephane Dupont (UMONS), Francesca Pozzi (CNR), Francesca Dagnino (CNR), Selami Ciftsi (TT), Leontios Hadjileontiadis (AUTH), Vasileios Charisis (AUTH), Marius Cotescu (ACAPELA), Athanasios Manitsaris (UOM), Maria Lazaridou (UOM), Christina Volioti (UOM), Sotiris Manitsaris (ARMINES).

Project ref. number ICT-600676

Project title i-Treasures - Intangible Treasures – Capturing the Intangible Cultural Heritage and Learning the Rare Know-How of Living Human Treasures



Editor Filareti Tsalakanidou (CERTH)

EC Project Officer Marcel Watelet

Abstract This report presents in detail the objectives and the main S&T results and achievements of the i-Treasures project. In addition, the potential socio-economic impact is discussed and the actions for the dissemination and use of i-Treasures are outlined.

Keywords Project context, project objectives, project S&T results, impact, dissemination, exploitation.



Signatures

Written by Responsibility- Company Date

Filareti

Tsalakanidou CERTH 12/06/2017

Verified by

Nikos

Grammalidis WP1 Leader (CERTH) 14/06/2017

Approved by

Nikos

Grammalidis Coordinator (CERTH) 14/06/2017

Kosmas

Dimitropoulos Quality Manager (CERTH) 14/06/2017



Table of Contents

1. Executive Summary ..............................................................................................9

2. Project context and objectives ............................................................................11 2.1 The challenge ..............................................................................................11 2.2 The goal.......................................................................................................11 2.3 The objectives..............................................................................................13

3. Description of main S&T results/foregrounds .....................................................17 3.1 WP2: Requirements Identification and System Specification ......................17

3.1.1 Review of the state of the art initiatives in the ICH field .................................. 17 3.1.2 Identification of the i-Treasures user requirements ......................................... 18 3.1.3 System specification ........................................................................................ 19

3.2 WP3: ICH Capture and Analysis..................................................................20 3.2.1 Facial expression analysis and modelling ....................................................... 21 3.2.2 Body and gesture recognition.......................................................................... 22 3.2.3 Electroencephalography (EEG) analysis......................................................... 25 3.2.4 Vocal tract sensing and modelling................................................................... 26 3.2.5 Sound processing ............................................................................................ 28 3.2.6 Data collection ................................................................................................. 29

3.3 WP4: Data Fusion and Semantic Analysis ..................................................31 3.3.1 Multimodal analysis and data fusion for semantic media interpretation.......... 31 3.3.2 Mapping sound to gestures and emotions and development of Intangible

Musical Instrument........................................................................................................... 34 3.3.3 ICH indexing by stylistic factors and locality variations ................................... 35

3.4 WP5: The Integrated Platform for Research and Education........................37 3.4.1 Educational scenarios and tools...................................................................... 37 3.4.2 3D visualisation module for sensorimotor learning.......................................... 39 3.4.3 Text-to-Song synthesis module ....................................................................... 42 3.4.4 Web platform for research and education ....................................................... 43

3.5 WP6: System Demonstration.......................................................................44 3.5.1 Demonstration for the traditional singing use case use................................... 45 3.5.2 Demonstration for the dance use case............................................................ 47 3.5.3 Demonstration for the pottery use case use.................................................... 50 3.5.4 Demonstration for the contemporary music composition use case use.......... 50 3.5.5 Demonstration challenges and conclusions .................................................... 51

3.6 WP7: Technical Assessment and Evaluation ..............................................51 3.6.1 Laboratory testing ............................................................................................ 52 3.6.2 Use cases evaluation....................................................................................... 53 3.6.3 Technical assessment of the system............................................................... 54



4. Potential impact, dissemination and exploitation ................................................56 4.1 Impact ..........................................................................................................56 4.2 Dissemination activities ...............................................................................58

4.2.1 Project logo...................................................................................................... 58 4.2.2 Project website ................................................................................................ 59 4.2.3 Social media .................................................................................................... 59 4.2.4 Brochure .......................................................................................................... 60 4.2.5 Posters............................................................................................................. 60 4.2.6 i-Treasures dissemination videos .................................................................... 61 4.2.7 Press releases and TV Interviews ................................................................... 61 4.2.8 Organisation of live ICH performances............................................................ 63 4.2.9 Organisation of conferences, workshops, special sessions and other events for

the scientific community................................................................................................... 64 4.2.10 Organisation of events for the general public .................................................. 65 4.2.11 Participation in events...................................................................................... 66 4.2.12 Organisation of the i-Treasures Workshop...................................................... 69 4.2.13 User group ....................................................................................................... 70 4.2.14 Clustering activities.......................................................................................... 70 4.2.15 Local training workshops ................................................................................. 72 4.2.16 Publications ..................................................................................................... 73

4.3 Exploitation ..................................................................................................83

5. Conclusions ........................................................................................................87

6. References..........................................................................................................91



List of Abbreviations

Abbreviation Meaning

2D Two Dimensional

3D Three Dimensional

API Application Programming Interface

AU Action Unit

ASM Active Shape Model

BGDCAM Body and Gesture Data Capture and Analysis Module

CMS Content Management System

CNN Convolutional Neural Network

DNN Deep Neural Network

DTW Dynamic Time Warping

EEG Electro-EncephaloGram

EEMD Ensemble Empirical Mode Decomposition

EGG Electroglottography

EMG ElectroMyoGraphy

FL Fuzzy Logic

HBB Human Beat Box

h-LDS High-Order Linear Dynamical System

HMM Hidden Markov Models

ICH Intangible Cultural Heritage

ICP Iterative Closest Point

ICT Information and Communication Technology

IMI Intangible Musical Instrument

IPR Intellectual Property Right

LDS Linear Dynamical System



Abbreviation Meaning

LHT Living Human Treasures

LMS Learning Management System

MEBN Multi-Entity Bayesian Network

NACE Nomenclature of Economic Activities

OSC Open Sound Control

PI Performance Index

PMD Photonic Mixer Device

PP Pedagogical Planner

RLMS Regularized Landmark Mean-Shift

SRH Summation of Residual Harmonics

SSH Summation of Speech Harmonics

STFT Short-Term Fourier Transform

STL STereoLithography

TCP/IP Transmission Control Protocol/Internet Protocol

TEL Technology Enhanced Learning

TPI Technical Performance Index

TTS Text-to-Song

UDP User Datagram Protocol

USB Universal Serial Bus

WP WorkPackage

XML Extensible Markup Language



1. Executive Summary Cultural heritage is not limited to monuments and collections of artifacts. It also includes fragile intangible traditions and living expressions that involve performances, practices, knowledge and skills such as music, dance, singing, theatre, and craftsmanship. These manifestations of human intelligence and creativeness constitute our Intangible Cultural Heritage (ICH). The importance of ICH is not limited to its cultural manifestation; it rather lies in the wealth of knowledge transmitted through it from one generation to the next.

The i-Treasures project provides a novel holistic approach for the safeguarding and transmission of ICH that goes beyond the common digitisation of ICH content. Its main contribution is the creation of new knowledge and novel cultural content by introducing a novel methodology for capture, analysis and transmission of ICH, based on cutting-edge ICT technologies. Our approach focuses on four ICH use cases, where the human body is dominant, i.e., traditional singing, traditional and contemporary dance, traditional pottery, and contemporary music composition, and is implemented through an open and extendable platform for ICH research and education, which enables the wide participation of communities, groups and individuals in the safeguarding of ICH.

A multi-sensory approach is adopted for the capture of different ICH expressions, based on 2D cameras and 3D sensors for body and face capturing, inertial and depth sensors for body or hand gestures capturing, EEG sensors for brain activity recording, and a novel hyper-helmet integrating various sensors for capturing the vocal tract (voice, tongue, lips, nasal vibrations, breathing, EGG signals). The data collected by these sensors are automatically analysed using intelligent computer vision and pattern recognition algorithms as well as data fusion techniques to identify basic ICH elements (e.g., postures, gestures, audio patterns, affective states). The extracted low or medium level features are further analysed using semantic analysis techniques to extract high-level metadata, easily understood by humans, which allow indexing and retrieval of ICH performances but also discovery of hidden knowledge. ICH performances and high-level metadata compatible with Europeana can be accessed through the Web-platform’s Digital Repository, which supports advanced search functionalities that exploit semantic analysis information.

To support the creation of innovative educational interventions, especially for those ICHs where traditionally no formal teaching/learning practices exist, a novel methodology and tools for ICH education have been proposed and exemplar educational scenarios were prepared in collaboration with various ICH experts, covering different learning situations and target populations. The educational scenarios were composed of ‘pedagogical plans’, that have been then implemented into ‘courses’ (delivered to users through the Learning Management System of the Web-platform), which offer a wide variety of possible learning activities from self-study, based on audio-visual material, to practice activities, based on novel game-like applications. The latter exploit recent advances in Web-based game engines and multi-sensory technology to offer an innovative and engaging virtual environment, which allows self-training and real-time evaluation of the learner’s performance by means of sensorimotor learning. Moreover, a Text-to-Song Web-interface was also created as an educational tool that allows the user to enter text and/or notes and produce the equivalent traditional singing voice. Finally, a novel intangible musical instrument has also been designed as a natural user interface that allows real-time sound synthesis based on gestures and emotions, aiming to address the problem of the long learning curve when accessing the musical ICH via tangible musical instruments.



The i-Treasures Web-platform for ICH research and education was successfully demonstrated to different populations in several European cities, including students of all ages, ICH practitioners and apprentices, and general public, based on a variety of demonstration scenarios, ranging from structured learning paths followed by school students to online courses for ICH beginners, to events for the general public. Moreover, the developed tools, together with classic ethnographic methods, have already been used by the cultural heritage community and the educational community for the implementation of small safeguarding projects, in the context of local training workshops for ICH preservation. User feedback suggests that the developed technologies constitute an innovative and promising set of tools that respect ICH tradition and have the potential to significantly contribute to its safeguarding and transmission, by taking advantage of recent advances in ICT technologies to analyse ICH content in a deeper and more effective way and to boost ICH education in ways that make it more appealing and attractive to young generations.



2. Project context and objectives i-Treasures “Intangible Treasures - Capturing the Intangible Cultural Heritage and Learning the Rare Know-How of Living Human Treasures” (FP7-ICT-2011-9-600676-i-Treasures) is an Integrated Project (IP) of the European Union's 7th Framework Programme ICT for Access to Cultural Resources. i-Treasures started on February 1, 2013, and lasted 51 months.

2.1 The challenge Cultural heritage is not limited to monuments and collections of artifacts. It also includes fragile intangible traditions and living expressions inherited from our ancestors, which involve performances, practices, knowledge and skills. Examples of such expressions include music, dance, singing, theatre, human skills, and craftsmanship. These manifestations of human intelligence and creativeness constitute our Intangible Cultural Heritage (ICH). The importance of ICH is not limited to cultural manifestation; it rather lies in the wealth of knowledge transmitted through it from one generation to the next [1].

ICH creations are transmitted orally or by gestures and are modified over a period of time through a process of collective recreation. As a consequence, there is always a risk that certain elements of ICH could die out or disappear if they are not treasured. The issue of the preservation of ICH came to light as the effects of globalisation caused the diminishment of the unique culture of many communities. As the world becomes more interconnected, many different cultures come into contact and communities start losing important elements of their ICH, while the new generation finds it more and more difficult to maintain the connection with the cultural heritage treasured by their elders.

In recent years, ICH has received international recognition, and its safeguarding has become one of the priorities of international cooperation mainly thanks to UNESCO's initiatives [1]. Museums can play the role of a mediator to fill the gaps between generations and share community knowledge; however, there are significant limitations in their role mainly due to the technologies employed, which usually aim at the digitisation of cultural content. ICH, however, is more efficiently preserved 'with' the people or community by protecting the processes that allow traditions and shared knowledge.

To this end, modern Information and Communication Technologies (ICT) can be employed to improve the presentation and transmission of ICH, raise public awareness, provide seamless and universal access to cultural resources, support services for research and education and bring hidden intangible treasures to light. Especially the latter constitutes the most challenging issue in the safeguarding of ICH and can a) give rise to a deeper understanding of this kind of cultural heritage, b) unveil unknown correlations between ICH treasures associated with societies from different time periods or geographical areas, c) create new ways for cultural expression that connect the past and have relevance in the contemporary world, and d) break new ground in education, knowledge transfer and research of ICH.

2.2 The goal The main objective of i-Treasures is to develop an open and extendable platform to provide access to ICH resources, enable knowledge exchange between researchers and contribute to the transmission of rare know-how from Living Human Treasures (LHT) to apprentices. To this end, the project aims to go beyond the mere digitisation of cultural content. Its main contribution is the creation of new



knowledge by adopting novel methodologies and new technological paradigms based on multi-sensory technologies for the analysis and modelling of traditional and contemporary ICH expressions.

Four different ICH use cases, where the human body is dominant, are studied: a) rare traditional songs, b) rare dance interactions, c) traditional craftsmanship, and d) contemporary music composition (Figure 1). For the first use case, the “Cantu in Paghjella” singing tradition of Corsica (France), the “Canto a Tenore” pastoral songs from Sardinia (Italy), and Byzantine hymns from Greece are studied; also, the “Human Beat Box” (HBB), a newly expanding contemporary singing style where the vocalist imitates percussive and instrument sounds. The dance case study focuses on traditional dances from Greece (Tsamiko), Romania (Căluş) and Belgium (Walloon), as well as on contemporary dance and Salsa dance. For the traditional craftsmanship use case, the wheel-throwing pottery is studied. Finally, the case study of contemporary music composition aims to develop a novel intangible musical instrument, which maps in real time information from the performer (both from his/her natural movements/gestures and from his/her brain activity) to music segments.

The core of i-Treasures lies in the identification of specific media patterns (e.g., postures, gestures, audio patterns, actions, affective states) in ICH performances using multi-sensor technology (2D/3D optical sensors, inertial sensors, ultrasound sensors, microphones, electroencephalograms (EEG), etc.), in order to reveal hidden intangible treasures. High-level semantics are extracted that could enable researchers to identify possible implicit or hidden patterns and correlations between different ICH expressions or different interpretation styles of the same ICH.

Figure 1: The i-Treasures ICH use cases (and corresponding sub-use cases): a) traditional and contemporary singing, b) traditional and contemporary dance, c)

traditional pottery, and d) contemporary music composition.

Data obtained from capturing and analysing ICH, coupled with other cultural resources, are accessible through the i-Treasures Web-platform, which allows



knowledge exchange among researchers, as well as education of new apprentices. User’s training and evaluation is realised through an embedded platform that adopts sensorimotor learning techniques, based on 3D visualisation technology and game engines. The platform employs high-quality avatars to visualise user and expert movements (also their vocal tract articulation) in engaging 3D environments, based on inputs from multiple sensors. Several educational scenarios are designed and different learning approaches for different target populations are adopted (

Figure 2).

Figure 2: The i-Treasures system.

2.3 The objectives To main S&T objectives of i-Treasures are the following:

• Identification of user requirements and system design

o Extensive review of state-of-the-art techniques/technologies/ systems/projects aimed to provide innovative and integrated solutions to allow access to cultural resources for the safeguarding and transmission of ICH.

o Establishment of user-expert group including experts, users and organisations related to cultural heritage and, especially, to the four selected ICH uses cases.



o Design and launch of an international survey involving the above mentioned user-expert group to define user requirements with a participatory approach.

o Identification, study and analysis of the specific characteristics of the selected ICH use cases in cooperation with the user-expert group.

o Definition of a set of requirements covering all the technological components of i-Treasures and subsequent iterative revisions.

o Specification of system architecture and interfaces based on user and system requirements.

o Design of an open and extendable platform for access to cultural resources.

• ICH capture and analysis

o Capture, processing and analysis of different forms of ICH, based on LHT’s knowledge.

o Development of multi-sensor set-ups for synchronised capturing of different modalities.

o Development of facial analysis algorithms for facial feature tracking and recognition of basic facial muscle movements, based on 2D and 3D image streams during singing.

o Development of body motion analysis algorithms for real-time capture, tracking and recognition of body gestures, hand gestures and articulated finger motions during an ICH performance for the dance, pottery and contemporary music composition use cases.

o Design and development of novel light-weight “hyper-helmet” including multiple sensors for the collection of different vocal tract articulatory signals during singing performances.

o Development of module for vocal tract signal analysis and modelling for singing.

o Development of new algorithms for tongue contour tracking and tongue movement visualisation.

o Development of electroencephalogram (EEG) data capture and analysis module for the recognition of the performer’s affective state in contemporary music composition.

o Development of sound analysis algorithms for the recognition of particular singing sounds and styles and the evaluation of a singer’s performance.

o Collection of data (2D and 3D image sequences, skeleton data, EEG signals, ultrasound images, sound signals, etc.) from different modalities and sensors for each use case; these are used for algorithm development and evaluation.

• Data fusion and semantic analysis

o Development of novel algorithms for multimodal analysis and fusion of data captured during ICH performances.

o Development of novel algorithms for semantic metadata extraction from ICH performances using information across media.

o Design and development of a novel intangible musical instrument, i.e., a natural user interface for performing and composing music based on



gestural and emotional expression. The upper body and finger gestures as well as the affective state of the user are captured by multiple sensors and are consequently mapped to various sound parameters, resulting in real-time sound synthesis.

o Development of novel algorithms for isolating the stylistic factors and locality variations of ICH performances.

• Integrated platform for research and education

o Definition of a novel methodology for ICH education and development of a tool for the design of innovative educational interventions in the ICH field.

o Identification and definition of the educational scenarios to be supported by the i-Treasures platform, composed of ‘pedagogical plans’ and related courses for each ICH considered by the project.

o Development of 3D visualisation platform for sensorimotor learning, i.e., a game-like interactive environment aiming to support people in learning or mastering different types of ICH. The platform will: a) support different sensors, b) use 3D technologies for body, face and vocal tract movement visualisation, c) support different educational scenarios for each ICH sub-use case, and d) allow the evaluation of the user’s performance and the provision of feedback about his/her mistakes.

o Development of algorithms for Text-to-Song synthesis (TTS), i.e., synthesis of singing voice based on lyrics and musical notes. Recording of singing voice corpora for traditional singing.

o Design and development of the i-Treasures Web-platform, including a) Content Management System (CMS), b) Learning Management System (LMS) supporting a set of exemplar courses for ICH learning and various user roles, c) Digital Repository for ICH resources, d) basic and advanced search functionalities for CMS, LMS and Digital Repository content.

o Integration of the different modules of i-Treasures into an operating Web-platform. Also, development of required middleware for component interoperability, implementation of modules’ communication, and access to ICH repository.

• System demonstration

o Definition of specific implementation and operational plans for the demonstration of the i-Treasures platform in the context of the selected ICH sub-use cases.

o Organisation of several activities for the demonstration of the integrated platform and of its main components to real users and experts in the context of all four use cases, based on the aforementioned demonstration scenarios.

o Collection of data for system evaluation.

• System evaluation

o Design of a methodological framework for assessing the performance of the integrated platform in terms of covering the user requirements and expectations.

o Laboratory testing of system components and functionalities.



o Evaluation of the performance of the i-Treasures platform and its components in real conditions with real users. Organisation of trials/tests for the four ICH use cases.

o Overall technical assessment of the i-Treasures system performance and validation against the user requirements.

• i-Treasures dissemination and exploitation

o Design of plan for the use and dissemination of the i-Treasures project.

o Dissemination of project objectives and results via the project website, social media accounts, project brochure, media releases and interviews.

o Release of promotional videos demonstrating the i-Treasures objectives and technologies.

o Organisation of special sessions, conferences and workshops to inform the scientific community about i-Treasures research and achievements.

o Organisation of dedicated Workshop to demonstrate and disseminate final project results.

o Attendance of conferences, workshops and other events addressed to the scientific community and the general public, covering topics related to i-Treasures research.

o Publications in international journals, conference proceedings, and books.

o Clustering with other research projects and organisations related to cultural heritage and ICT.

o Establishment of an active group of ICH experts.

o Organisation of local training workshops addressed to education and culture professionals as well as local community members with an interest in heritage preservation, new technologies and local development, aiming to equip participants with knowledge and skills that will allow them to design and implement activities for ICH safeguarding.

o Market analysis and development of strategy for exploiting project results beyond the life of the project.



3. Description of main S&T results/foregrounds In the context of the i-Treasures project, an open and extendable platform for ICH research and education has been developed [2],[3]. The platform integrates various components for ICH capture, analysis, and learning/transmission, focusing on four ICH use cases, where the human body is dominant: a) traditional singing, b) traditional and contemporary dance, c) traditional pottery, and d) contemporary music composition. A multi-sensory approach is adopted for the capture of different ICH expressions, based on 2D and 3D cameras for body and face capturing, inertial sensors for body or hand gestures capturing, EEG sensors for brain activity recording, ultrasound sensors for vocal tract capturing, etc. The signals and measurements collected by these sensors are automatically analysed using intelligent computer vision and pattern recognition algorithms as well as data fusion techniques. The low or medium level features extracted are further analysed using semantic analysis techniques to extract high-level metadata, easily understood by humans, which allow indexing and retrieval of ICH performances. A set of educational tools including pedagogical plans, e-courses and serious games for ICH practicing have been developed for ICH education and transmission of rare-know-how from ICH experts to learners.

In the following, the main science and technology results of the i-Treasures project are summarised per workpackage (WP).

3.1 WP2: Requirements Identification and System Specification The main S&T results of WP2 are a) the review of the state-of-the-art techniques/ technologies/ systems/ projects, b) the identification of user requirements for an integrated system for ICH research, education and transmission, and c) the specification of the architecture, components and interfaces of the i-Treasures Web-platform.

More details can be found in the WP2 deliverables: D2.1 “First report on user requirements identification and analysis”, D2.2 “First report on system specification”, D2.3 “Second report on user requirements identification and analysis”, D2.4 “Final report on system specification” and D2.5 “Final report on user requirements identification and analysis”.

3.1.1 Review of the state of the art initiatives in the ICH field

An extensive state-of-the-art review on existing technologies and projects related to i-Treasures was delivered in the early months of the project. The aim was to review the scientific literature concisely and gather instances of practical experience on applications and projects providing access to cultural heritage, as well as on other important aspects related to the safeguarding and transmission of ICH. The emphasis was on capturing the essential elements of prior projects and experience with a view to identify their relevance to i-Treasures.

More specifically, the survey highlighted recent efforts to safeguard ICH following the adoption of the relevant convention by UNESCO [1], paying particular attention to the aims, objectives and methodology of state of the art. A detailed investigation of recent projects aimed at safeguarding ICH on both international and national levels was also included, consisting of two parts. The first explored the work of UNESCO in this field by providing an overview of its most prominent projects. The main conclusion is that UNESCO has adopted a primarily archival and encyclopaedic approach. The second part introduced the topic of community participation as key to the safeguarding of intangible heritage and examined activities undertaken at a



more grassroots level. As opposed to purely archival projects, most of these activities are mainly focused on processes of transmission and dissemination among practitioners and the new generation [4],[5]. In addition, an overview of the different technologies considered in i-Treasures was presented, including an introduction to these technologies (discussing what, how and why), an examination of state-of-the-art sensors, tools, algorithms and projects (also identifying gaps and constraints) and a discussion of possible use of these technologies for ICH preservation [5]. More details can be found in D2.1, Section 4.

3.1.2 Identification of the i-Treasures user requirements

The requirements definition process was iterative and based on a participatory approach [4],[6],[7],[8]. An international group of more than 200 ICH experts, performers and users related to the selected ICH expressions (Canto a Tenore, Cantu in Paghjella, Byzantine music, Human Beat Box, Tsamiko dance, Calus dance, Walloon dance, contemporary dance, pottery, and contemporary music composition) was actively involved, through surveys and interviews, in the complex tasks of identifying specificities of rare traditional and contemporary know-how, discovering existing teaching and learning practices, and, finally, identifying the most cutting-edge technologies able to support innovative preservation and learning approaches to ICH [9],[10],[11] (see D2.1, Section 5). A first list of user/system requirements was elaborated based on overview of past/ongoing projects in the field of ICH preservation and transmission, literature review, user questionnaire analysis (10 questionnaires – one for each sub-use case – have been released covering several aspects of ICH expressions, i.e., physical, emotional, social, knowledge, teaching/learning, etc.), and interviews with selected stakeholders (see D2.1, Sections 6 and 7) (Figure 3).

Figure 3: Definition of user and system requirements.

Later, the level of accomplishment/implementation of the initial user/system requirements was assessed through an iterative process that occurred after the completion of the first development phase. This process allowed the identification of the aspects of the developed technologies that needed to be improved and resulted in an updated, more comprehensive and exhaustive list of requirements (see D2.3).



After the completion of the development of the system, users’ requirements underwent a further process of revision. To this aim, in line with the approach taken in the previous phases, the approach chosen was to integrate information coming from three different groups of stakeholders: users, experts and developers. The global evaluation of the requirements was positive and some improvements were suggested for the different modules. The resulting list of conclusive recommendations and indications for future work remains as a as sort of a legacy for future projects addressing the same area of research (see D2.5).

3.1.3 System specification

Based on the requirements, a detailed system specification was devised, resulting in a robust architecture design. The latter is open and modular and aims to support different applications and tools (Figure 4). It comprises of five main components: i) the data capture and analysis module, which supports a wide variety of sensors and state-of-the-art technologies in motion, emotion and sound capture, analysis and recognition; ii) the data fusion and semantic analysis module, which extracts medium and high-level features based on the output of the previous module and unveils hidden knowledge and correlations; iii) the 3D visualisation platform for sensorimotor learning, which supports users to learn, practice and master the different types of ICH through game-like applications; iv) the Text-to-Song synthesis module, which provides an interface that enables the user to enter text and/or music notes and the system to produce the equivalent singing voice supplementing for the absence of a teacher; and v) the integrated Web-platform for ICH research and education, which glues together the different bits and pieces of i-Treasures and wraps them with an one-stop user friendly environment, available for everyone to access. System specifications have also been updated, after the completion of the first development and evaluation phase, to comply with the updated functional system requirements and accurately reflect the architecture of the developed system and sub-systems (see D2.2 and D2.4).



Figure 4: i-Treasures system architecture.

3.2 WP3: ICH Capture and Analysis WP3 focused on the capture and analysis of ICH performances with the use of multi-sensing technologies including optical sensors, depth sensors, inertial sensors, electroglottograph (EGG) sensors, electroencephalogram (EEG) sensors, ultrasound sensors, etc. Based on the in-depth analysis of the various cultural expressions as well as discussions with ICH experts, several algorithms and have been developed for the analysis of ICH performance elements. These include modelling and recognition of facial muscle movements, body movements, hand and finger gestures, brain activity, vocal tract operation (including tongue and lips movement), acoustic speech and music sounds. Multi-sensor calibration and synchronisation protocols have also been designed to allow synchronous recording of different ICH elements (e.g., body movements and singing voice) and novel sensor fusion techniques have been implemented, which combine data captured by different sensors in a common augmented structure (e.g., fusion of data from multiple depth and inertial sensors into a single full-body skeleton). A prototype hyper-helmet embedding several sensors for vocal tract capture has been designed. Finally,



numerous recordings of different ICH expressions using multiple sensors have been realised in cooperation with various ICH experts. Part of the collected ICH data is available to other researchers via the project Website1.

More details can be found in the WP3 deliverables: D3.1 “First report on ICH capture and analysis”, D3.2 “First version of ICH capture and analysis modules”, D3.3 “Final report on ICH capture and analysis”, and D3.4 “Final version of ICH capture and analysis modules”.

3.2.1 Facial expression analysis and modelling

A robust, unobtrusive face tracker, which has the ability to analyse 2D and 3D facial images in real time and recognise basic facial movements was developed by CERTH (see D3.1, Section 3.2 and D3.3, Section 3.2). The input of the tracker is a stream of registered colour (2D) and depth (3D) images of the singer’s face captured by a Kinect sensor [12]. Data processing is implemented as a four step-procedure: i) First, the position and 3D pose of the face are estimated based solely on 3D data. ii) Then, the positions of a set facial landmarks corresponding to the boundary of the face and important facial features (eyebrows, eyes, nose, and mouth) are tracked based on prior knowledge of facial geometry and model fitting methods like Active Shape Models (ASM) and Regularized Landmark Mean-Shift (RLMS), which are extended to handle depth data. The facial tracking result is subsequently enhanced by the use of local feature detectors aiming to improve the tracking accuracy of individual features such as the mouth or eyebrows. New techniques have been proposed to this end, which have shown to obtain greater accuracy, especially in the case of open mouth facial expressions. iii) Next, a set of measurements - geometrical, texture and curvature descriptors - is extracted in order to effectively model the changes in the shape of facial features and their geometrical arrangement as well as deformations of the face surface caused by wrinkles or furrows. iv) These measurements are subsequently used to identify basic facial muscle movements called facial action units (AU) (e.g., eyebrows raised, lip corners pulled up, jaw drop, etc.). The proposed methodology provides increased accuracy compared to state-of-the-art techniques.

An open source 3D head talking model has also been integrated to visualise the AU recognition results. Simple animations have been implemented based on the translation of the detected AUs (and corresponding measurements) into facial animation parameters supported by the model (Figure 5).

Figure 5: Facial expression analysis software interface.

11 i-Treasures open datasets: http://www.i-treasures.eu/content/datasets



3.2.2 Body and gesture recognition

Several algorithms and tools for body motion tracking/recognition and hand/finger gesture recognition were developed and were employed for body motion capture and analysis in different ICH sub-use cases. These are summarised in the following:

• Body tracking

o BGDCAM (Body and Gesture Data Capture and Analysis Module): A tool for human skeleton tracking based on fusion of multiple skeletal streams captured by multiple (up to four) Kinect II sensors [12]. The sensors are calibrated using the Iterative Closest Point (ICP) algorithm on the skeletal joint positions. The skeletal data are fused by combining the outputs of the sensors to reduce occlusion and self-occlusion problems and to increase the area of coverage of the motion capture space [14],[15],[16],[17]. This tool is appropriate for capturing a single person moving along a line or an arc (e.g., a single dancer in Tsamiko dance) (Figure 6).

The tool also supports the synchronised capturing of hand animation data from Leap Motion sensors [13] and body animation data from multiple Kinect II sensors for the pottery use case. Leap motion and Kinect data are fused in order to output a single stream containing skeletal information of both the hands/fingers and the torso of a potter. Calibration of sensors is performed based on wrist and elbow joint positions, captured by both types of sensors (see D3.3, Section 3.1.1.2).

Figure 6: Tsamiko dance recording using the BGDCAM tool. The dancer’s movements are captured by three depth sensors (right). A fused human skeleton is produced by combining the three skeletal streams produced by the depth sensors (left).

o ‘sar’: A tool for the calibration and synchronisation of multiple depth cameras

(Kinect II sensors) for the recording of couple dances or cyclical group dances like Walloon. The tool fuses the partial and noisy depth maps produced by the depth sensors to provide a single full-body depth map, without missing data, for each dancer (see D3.1, Section 3.1.1.2).

o ITSkeletonFusion: A methodology for upper body data capture and visualisation based on fusion of the outputs of inertial sensors (Animazoo), Kinect and Leap Motion. Sensor data are streamed in OSC format. To fuse the point clouds (skeletons) captured by different sensors, fusion rules are applied, which define how joints from sensor A and sensor B should be coupled together. Usually, a single rule is used (e.g., left hand with left palm), but in some cases several rules may be required (e.g., left hand with left palm, and right hand with right palm). This module is used in the intangible musical instrument to provide a single fused skeleton of the upper



body of the performer including torso, hands and fingers (see D3.3, Section 3.1.3.2.1).

• Body gesture recognition / feature extraction

o An algorithm for the recognition of the basic motion patterns of Tsamiko dance based on the fused skeletal stream provided by the BGDCAM tool. The skeleton is transformed to achieve view invariant detection and is split into parts. For each part, we detect a sequence of postures displayed. A codebook of postures is constructed in advance, and, thus, each motion is transformed into a sequence of words of the codebook. These sequences are used to train a classifier to recognise specific motion sequences such as dance moves [14],[15] (see D3.1, Section 3.1.2.1).

o An algorithm for the segmentation of dance sequences and the recognition of dance steps. Dance periods are detected based on the displacement of the waist of the dancer. Dance steps segmentation is performed by locating local maxima in foot joint trajectories [15],[17]. Several medium-level features are extracted for each step, describing the step execution (foot id, foot direction, foot position, etc.) (see D4.1, Section 5.2.1.2).

o A methodology for extracting dynamics from multi-dimensional time-evolving data, such as skeletal motion capture data, using a bag of Higher-Order Linear Dynamical Systems (h-LDS) [18],[19]. The proposed method can be used for the analysis of human motion in various applications such as the recognition of dance figures or pottery gestures (see D3.3, Section 3.1.2.1).

o A software library for real-time recognition and tracking of body gestures based on several implementations of the Hidden Markov Models (HMMs) [20],[21],[22],[23] (see D3.1, Section 3.1.2.2).

o iHMM: An HMM-based body gesture recognition module based on the MotionMachine framework (presented below as part of WP4). The HMMs are trained using high quality motion data from expert performance recordings captured by a high-precision mocap sensor; in the second phase, a small amount of data recorded by a low-cost and less precise sensor like Kinect is used to adapt the trained models and create, in this way, a system able to recognise gestures recorded by this low-cost sensor [24],[25]. This module has been used for the recognition of basic gestures of Walloon dance (Figure 7). Its main advantage is that it allows recognizing gestures independently of the sensors used for data capture (see D3.3, Section 3.1.2.3 and D3.4, Section 3.1.5).



Figure 7: Interface of iHMM module for recognizing body gestures using HMM models.

o iDTW: Body gesture recognition and evaluation module based on Dynamic

Time Warping (DTW) and relational features developed based on the MotionMachine framework. The module computes a motion template by computing the mean of the relational features based on several executions of the same gesture and, then, measures the similarity between the template and some movement or a continuous motion sequence to recognise the gesture (see D3.3, Section 3.1.2.4 and D3.4, Section 3.1.6)

• Hand/finger motion detection and recognition

o x2Gesture (eXpert eXpressive Gesture): A 3D hand gesture recognition engine, inspired by the Gesture Variation Follower, which aims at recognizing musical or technical expert gestures in real-time, taking also into account gesture expressive variations. It supports both learning and performing of gestures as a unified user experience. It has been used in both the contemporary music composition and the pottery use cases [26],[27],[28] (see D4.4, Section 6).

o Algorithm for pottery scene segmentation and object profile extraction [29]. The algorithm segments the pottery scene, which consists of the turning plate, the clay/object and the potter’s hands and fingers, based on the 3D data provided by two PMD cameras [30] (Figure 8). Assuming that the plate is initially empty, the 3D cloud corresponding to the clay/object and the position of the potter’s hands can be adequately estimated (see D3.3, Section 3.1.3.1).



Figure 8: a) Setup of depth sensors to capture the pottery scene (left), b) example of image segmentation (middle) and c) feature extraction result (the detected points correspond to the two elbows and the palms, thumbs, and centroids of the rest of the fingers of the two hands) (right).

3.2.3 Electroencephalography (EEG) analysis

An EEG-based affective state recognition module, based on a commercially available EEG headset, i.e., the Emotiv EPOC [31], has been implemented that, in its core, consists of a standalone application for data processing, classification and user interfacing. The modelling of affective states is based on the valence - arousal two-dimensional model, with valence denoting the polarity of an affective state (positive to negative) and the arousal reflecting the measure of excitation (high to low) of the state. In particular, the module implemented comprises a standalone .NET application for Windows that mainly: a) serves as the medium for capturing EEG signals streamed by the EPOC headset (14 recording channels); b) processes the captured EEG data based on a new algorithm for valence and arousal recognition; c) displays the result of each recognition cycle as a quartile of the valence-arousal two-dimensional space, part of the main user interface element (Figure 9). The application also provides basic controls (start/stop of streaming and recognition), simulation mode when the headset is not available, and it is UDP-based communication-ready for integration with other modules (see D3.4, Section 3.3).

Figure 9: Interface of the affective state recognition module (middle) based on the use

of an EEG headset (left). The recognition result is displayed as a quartile of the valence-arousal two-dimensional space (right).

The algorithm for affective state recognition was developed based on laboratory EEG data collection during emotion elicitation and comprises the following steps: i) from the streamed EEG signals (14 channels), four second-long epochs (recognition cycle) are stored in a buffer; ii) from the stored epochs, features corresponding to the low frequency component (1-7 Hz) of channels from bilateral fronto-central and



temporal regions as well as right parietal and occipital sites, derived using the Hilbert-Huang transform, are extracted for each of the four one-second intervals; iii) Each of the four feature vectors produced is fed to a pre-trained (using the previously mentioned laboratory EEG data) Support Vectors Machine (SVM) classifier that outputs the estimated valence (positive or negative); iv) The majority outputted class constitutes the valence level (positive or negative) attributed to the four second-epoch; v) the result is combined with the arousal level (high or low) derived after comparing the mean normalised (0-1) excitement level produced by the Emotiv EPOC proprietary algorithm over the four-second epoch with a 0.5 threshold. The pair of valence-arousal level is an estimate of the general affective state for the recognition cycle [32].

Eventually, the output of the module is a near real-time (every four seconds) estimate of the general affective state of the user. This stream of estimated affective states serves as input to the associated user interfaces in the contemporary music composition use case (see D3.3, Section 3.3).

3.2.4 Vocal tract sensing and modelling

A multi-stage methodology for the capture and analysis of the vocal tract during singing performances has been developed, based on multiple sensors mounted on a light-weight hyper-helmet [33]. The prototype hyper-helmet (Figure 10) consists of a suite of non-invasive sensors including ultrasound for tongue contour tracking, USB camera to capture lips and mouth movements, lapel microphone for voice recording, piezoelectric accelerometer mounted on the nasal bridge for detecting the nasal resistance, EGG to measure and record vocal fold contact movement during speech, and respiratory belt sensor to determine breathing modalities and position [34] (see D3.1, Section 3.4.2).

The vocal tract capture module can synchronously record and display the signals captured by the different sensors. Two other modules have also been developed for analysing and visualising the signals recorded by the hyper-helmet (see D3.3, Section 3.4.2).

o The i-THRec (i-Treasures Helmet Recording software) contains multiple graphical user interfaces that allow sensor calibration and operation monitoring, recording of singing performances and organisation/storage of recorded data, visualisation of current and old recording sessions [35],[36] (Figure 10). A network version of the i-THRec tool has been developed to support the recording of polyphonic singing performances (e.g., three singers in the case of Cantu in Paghjella or four singers in the case of Canto a Tenore). The tool ensures synchronous acquisition of three or four singers’ performances, with acoustical isolation of singers so that the microphone of one singer only acquires his/her voice, and provision of audio-visual feedback to the singers. This allows singers to sing together as a polyphonic group and to be recorded in more natural conditions.

o The i-THAn (i-Treasures Helmet Analysing) software for offline hyper-helmet data display, validation and analysis is used for the post-processing of vocal tract signals recorded via i-THRec, for both monophonic and polyphonic singing performances. The tool supports several algorithms for the automatic extraction and storage of a large set of features from the recorded signals such as acoustic, articulatory and physiological data to describe vocal tract behaviour during signing [36], [37],[38].

Several algorithms have been developed for the analysis of the various signals recorded by the hyper-helmet sensors [37],[38].



Figure 10: The i-THRec data acquisition tool for multi-sensor vocal tract capturing (right), based on the use of a prototype light- weight hyper-helmet (left) including: ultrasound probe sensor, optical camera, lapel microphone, nasality piezoelectric sensor, electroglottograph sensor, and respiration belt.

By far, the most challenging data are those derived from the ultrasound transducer that captures the tongue, since they are noisy, variable, and difficult to interpret. To this end, several approaches have been evaluated: i) an algorithm has been developed for tongue contour tracking in ultrasound images based on deep neural networks (DNNs) [39] (see D3.1, Section 3.4.5.1), ii) a new approach to contour tracking using “snakes” has been proposed, allowing to keep track of the tongue even in poor imaging conditions [40],[41] (see D3.3, Section 3.4.4.1). The extracted tongue contours are used to drive a 3D tongue model (used in a game-like application for HBB learning) in real-time, after the computation of the relationship between the positions of given points on the contour and the corresponding nodes of the 3D model [42],[43] (see D5.2, Section 4.1.3.5). The information about these “tissue points” of the tongue may not be accurate enough to obtain a stable 3D shape, so another approach has been investigated, based on building a database of mid-sagittal contours from 3D model-based tongue shapes [44]. Then, the contour extracted from the ultrasound image is compared to the contours in the database. The closest 3D contour is used to associate the 3D shape to the ultrasound image (see D5.5, Section 4.2.1.2). Finally, following the same idea, we have designed a classifier based on Convolutional Neural Networks (CNNs) to associate an ultrasound image to a tongue gesture without extracting the contour [45]. These two approaches allow producing stable 3D shapes, but they cannot yet be used for a smooth animation of the 3D tongue model (Figure 11).

Figure 11: Sample frames of 3D tongue modelling: The ultrasound images of the tongue (extracted tongue contour shown in green) and the corresponding 3D tongue

model shapes are shown.



3.2.5 Sound processing

Several algorithms have been developed for the analysis, detection and recognition of vocalisations in traditional and contemporary singing as well as for music sound analysis [46]. In addition to the collection and careful ground truth annotation of various new corpora for the four singing sub-use cases to support research and benchmarking, the following results have been achieved:

• Detection of note starts: note onset detection methods have been developed to cover a range of 18 variants of magnitude-based (including energy, log-energy domains, and their time derivatives) and Short-Term Fourier Transform (STFT)-based approaches. New combinations of previously proposed sub-blocks have actually been proposed through a modular redesign of the pipeline. It was applied to HBB singing where it enabled to detect new musical events (notes and drums) to an accuracy level similar of real instruments. For the other singing use cases, note start detection necessarily involves the measurement of note height (pitch) and, hence, the detection of pitch transitions, or STFT-based techniques able to detect new notes even when no changes in the overall energy are visible (common in singing). Singing performances of Byzantine music appeared very challenging being heavily expressive, with the use of ornaments, appoggiaturas (sometimes 3 half-steps up) and vibrato. A new pitch estimation post-filtering was designed relying on adaptive median filtering overweighting parts of pitch curve closer to stable pitches of the musical scale (see D3.3, Section 3.5.1.1).

• Estimation of note pitches: the four of the most representative state-of-the-art techniques for pitch extraction: YIN, RAPT, Summation of Residual Harmonics (SRH), and Summation of Speech Harmonics (SSH) have been used. A new hybrid SRH-SSH method has also been proposed, achieving better scores than the best performing individual methods taken separately (see D3.1, Section 3.5.2.2).

• Recognition of sound categories: our novel developments where focused on Human Beatbox (due to the specific nature of percussive sounds that need to be recognised). Statistical sequence modelling techniques similar to the ones used for speech recognition have been used. We discovered that their optimal settings differ a lot from those traditionally used for speech recognition: in particular the analysis needs to be much more frequent in time. Besides, a second empirical study showed that an extended feature set (including features resulting from the onset detection and pitch extraction process) and non-linear dimensionality reductions yield improved recognition performance (see D3.3, Section 3.5.1.3).

• Scoring system for singing use cases, involving the forced alignment of reference expert and learner performing the same exercise, used for the evaluation of the learner’s performance in the game-like applications for HBB and Byzantine music (see D3.3, Section 3.5.1.4).

• Music analysis using tempo estimation and beat tracking (to support the dance use cases): we first used existing state-of-the-art approaches for tempo and beat tracking. However, we had to design pre-processing techniques, as well as tune the parameters to make these provide good results on the selected sub-use cases, especially on Tsamiko, for which tempo often accelerates. We further improved the system using a pre-processing audio filter in order to attenuate solo instruments (wind instruments) and voice, which can be mixed very loudly in Tsamiko and, thus affect the tempo tracking. Besides, an algorithm for tempo estimation using ensemble empirical mode decomposition (EEMD) analysis has been proposed (see D4.3, Section 6.3).



3.2.6 Data collection

Numerous recordings of different ICH expressions using multiple sensors have been realised in cooperation with various ICH experts and learners for the different ICH sub-use cases (Figure 12). These are summarised in the following Table (see D3.1 and D3.3).

Table 1: ICH data collection for the different i-Treasures sub-use cases. Sub-use case Subjects & small description Data recorded/Sensors used

Tsamiko dance

Experts and students dancing different variations of the dance individually and together.

Skeletal streams, depth data and video recorded simultaneously by 3-4 Kinect sensors.

Calus dance Expert performing the basic steps and motifs of the dance and different styles.

Skeletal streams, depth data and video recorded simultaneously by 3-4 Kinect sensors.

Walloon dance

• Expert dancers and beginners performing basic steps of different styles of Walloon dance.

• Group of dancers performing choreographies.

Point cloud data or skeletal streams captured with 5-6 Kinect sensors and mocap data captured with the Qualisys optical motion capture system.

Contemporary dance

Expert dancers, each performing individually a common choreography, as well as improvisation on emotionally charged musical pieces (tagged with arousal and valence).

Mocap data (full skeleton) captured with the Qualisys optical motion capture system and video captured by camera.

Cantu in Paghjella

• Expert singer producing vowels in singing and speaking mode and connected CV syllables with major Corsican vowels and consonants and also singing Paghjella songs.

• Groups of 3 singers performing different songs (polyphonic singing - three voices).

• Vocal tract signals recorded with hyper-helmet (single singer).

• Synchronous recording of vocal tract signals with 3 hyper-helmets (three singers).

Byzantine music

• Expert singer producing vowels in singing and speaking mode and singing segments of Byzantine chants in two styles.

• Expert singing different hymns in different arctic tones.

• Vocal tract signals recorded with hyper-helmet.

• Microphone for voice, Kinect for facial data and ultrasound probe for tongue movement.

HBB

• Expert performing various songs including music instruments’ imitation, animals’ imitation, modern beat.

• Expert producing different sounds: individual drums sounds, rhythms, instruments and freestyle, in different tempi.

• Vocal tract signals recorded with hyper-helmet.

• Microphone for voice recording.

Singing (facial analysis)

Subjects displaying different facial muscle movements (facial action units).

2D+3D image streams recorded with Kinect.

Pottery Pottery expert performing the four basic phases of object creation to create different real objects.

Two setups: • Animazoo motion capture

suit with sensors placed on the upper part of the body, Kinect sensor for body



Sub-use case Subjects & small description Data recorded/Sensors used

skeleton and video capture. • PMD depth camera to

capture finger gestures and clay, MYO sensor to capture EMG signals, inertial sensors to capture wrists motion.

Pottery Pottery expert performing virtual pottery gestures (without using clay) creating different virtual objects.

Skeletal streams recorded by 2 Kinects (body & hands) and a Leap motion sensor (fingers). Video recorded by 3rd Kinect.

Contemporary music (musical gestures)

Pianists performing different musical gestures in different tempi, in real and virtual conditions: • real gestures playing the piano, • virtual gestures using the Intangible

Musical Instrument.

• Real gestures captured with Animazoo suit.

• Virtual gestures captured with Kinect (body), Leap motion (fingers), and Animazoo (wrists).

Contemporary music (Affective analysis)

Subjects targeting the evocation of four affective states (positive or negative valence and of high or low arousal) or emotion transition based on: • image and sound stimuli, • musical stimuli (excerpts of Beethoven

piano sonatas).

EEG signals recorded with the Emotiv EPOC headset.

Contemporary music (Affective analysis)

Subjects watching videos (covering transitions between affective states ) and rating their affective state: • dynamically, on a digitiser pad

representing the valence-arousal plane, by using a stylus,

• at the end, based on a five point-scale self assessment manikin (valence and arousal).

EEG signals recorded with the Emotiv EPOC headset and user feedback about his/her affective state.

Part of the collected ICH data is available to other researchers via the project Website2. This ICH dataset covers the majority of the selected ICH sub-use cases. It includes raw performance data captured by various sensors as well as accompanying annotation data and metadata. For each performance, a small description is provided including: a) overview with general information about the specific ICH sub-use case, b) detailed description of data recording (artists/experts involved, location, capture time, provider, etc.), c) experimental sensor set-up and protocol used for data recording, and d) detailed description of the recorded dataset files, including their data format [47].

2 i-Treasures open ICH dataset: http://www.i-treasures.eu/content/datasets



Figure 12: Data collection for different ICH sub-use cases using a variety of sensors.

3.3 WP4: Data Fusion and Semantic Analysis The main results of the research performed within WP4 are: a) a set of ontologies for the representation of the ICH domain knowledge corresponding to the selected sub-use cases, b) a new framework for multimodal semantic analysis of ICH performances based on Multi-entity Bayesian networks (MEBNs), c) a novel methodology for implicit or explicit mapping of sound to gestures, taking also into consideration the emotional status of the performer, d) a novel intangible musical instrument (IMI), which produces sounds based solely on the performer’s gestures and affective state, implementing the aforementioned methodology, e) an augmented music score that includes musical gestures and emotional annotations on top of musical notes, f) a library and a set of software tools for the stylistic analysis of ICH performances such as dance, and g) a tool for the visualisation and stylistic analysis of finger gestures in pottery based on electromyography signals.

More details can be found in the WP4 deliverables: D4.1 “First version of multimodal analysis, fusion and semantic media interpretation”, D4.2 “First version of mapping sound to gestures and motions”, D4.3 “Final version of multimodal analysis, fusion and semantic media interpretation”, D4.4 “Final version of mapping sound to gestures and motions” and D4.5 “Report on ICH indexing by stylistic factors and locality variations”.

3.3.1 Multimodal analysis and data fusion for semantic media interpretation

A set of ontologies for the representation of the ICH domain knowledge corresponding to the selected sub-use cases has been delivered based on a multi-stage approach, which builds on user requirements and discussions between ICH domain experts and technology developers. The knowledge encoded with these ontologies was acquired under a thorough investigation of the domain and, more specifically, by collecting the answers of ICH domain experts to carefully designed questionnaires (Figure 13). Thus, the ontologies were designed through a rigorous methodology, leading to accurate knowledge modelling and representation (see D4.1, Section 3).



Figure 13: Part of the graph of the ontology developed for Human BeatBox singing.

Also, at a later stage of this multi-stage approach, novel knowledge representation models, i.e., multi-entity Bayesian networks (MEBNs), and inference algorithms were investigated, in order to model the inherent uncertainty that is prevalent in the domain of ICH [48],[49] (see D4.1, Sections 4 and 5). The goal of these models is to encode the multimodal nature of the ICH content and combine the medium-level features that have been extracted by applying the WP3 analysis modules on different modalities (e.g., visual and auditory characteristics, such as the gestures and the body movements of the dancer performed while executing the dance and the beats of the music accompanying the dancer) based on the notion of rhythm. The use of rhythm as the underlying reference allowed us to evaluate the level of synchronisation between dance steps and music beats. The ability to combine these features allowed the extraction of high-level metadata, such as different ICH styles and variations, the assessment of the proficiency level of a performer, as well as the evaluation of the synchronisation across different modalities (see D4.3, Section 5). Moreover, an algorithm specifically designed for training the MEBN-based models was also developed. More specifically, a constrained version of the Expectation-Maximization algorithm was tailored to the procedure of training a MEBN, i.e., selecting specific values of the parameters utilizing example data (see D4.3, Section 4).

The potential of the proposed models to effectively capture and jointly evaluate multimodal information was verified in the Salsa dances) (Figure 14), but also in the context of problems related to composer identification. Indeed, experimental results



showed the efficiency of the models employed for dance performance evaluation, when compared to conventional models, in terms of analysis based on evidence with inherent uncertainty, and under the circumstances of having both certain and uncertain knowledge. More specifically, the experiments demonstrated the advantages and potential of these models in modelling this type of knowledge. In addition, a novel algorithm was designed for calculating sufficiently accurate values for MEBN model parameters, something which led to superior analysis results in terms of dancer assessment accuracy, compared to models having manually selected parameters (i.e., guessed by experts, since absolute and certain knowledge about the values of these parameters was not possible).

Figure 14: MEBN-based model for analyzing Salsa dance performances based on the

notion of rhythm (synchronisation between music beat and dance steps).

Using the above mentioned knowledge modelling framework, we have also designed a MEBN model for composer identification, i.e., classifying musical pieces of unknown origin taking as input their musical scores. The model was designed to capture musical knowledge, i.e., concepts such as the value and duration of a note as well as the concept of the chord (i.e., a simultaneous occurrence of multiple notes). The model, after training with example musical scores, achieved a high accuracy of recognizing the composer of a musical piece by taking as input its musical score. Efficient classification via sophisticated modelling is a very important topic, since it can lead to better understanding of the similarities and differences between the style characterizing different composers.



Lastly, we developed an audio restoration algorithm with the aim to safeguard and conserve traditional musical pieces and songs that have been recorded with imperfect means. To this end, the restoration algorithm was designed with the aim to remove specific types of noise appearing in such recordings, such as sounds produced due to dust and scratches that have damaged vinyl records.

3.3.2 Mapping sound to gestures and emotions and development of Intangible Musical Instrument

A novel intangible musical instrument (IMI) has been created aiming to address the problem of the long learning curve when accessing the musical ICH via tangible musical instruments, and facilitate access to the knowledge of music performers. To this end, novel technologies have been developed for analysing the performance of pieces of classical and contemporary composers (primarily focusing on piano-playing), transmitting knowledge related to the musical gestures performed, and introducing new ways for contemporary music composition [50],[51],[52],[53],[54] (see D4.2 and D4.4).

More specifically, the IMI offers a novel natural user interface for learning, performing and composing based on gestural and emotional expression. The upper body and finger gestures, as well as the emotional state of the user are captured by multiple sensors (one Kinect for body gestures, two Leap motion sensors for finger gestures and Emotiv EPOC for capturing EEG signals related to affective state), they are analysed and recognised, and they are implicitly or explicitly mapped to various sound parameters, resulting in real-time sound synthesis. The interface is supported by a novel set-up made of plexiglas, shaped so as to look like a table, over which the user can perform piano-like gestures (Figure 15). A portable set-up of the instrument, packaged in a wooden box, has also been designed.

Figure 15: Intangible musical instrument (IMI) set-up comprised of sensors for body and finger motion capturing as well as sensor for capturing brain activity. A plexiglas table renders the gestures more ergonomic: a) sensor set-up (left), b) user performing piano-like gestures using the IMI (middle), and c) portable set-up of the IMI (right).

Moreover, an "augmented music score" representation has been designed and integrated in the contemporary music composition (CMC) game. This score is an enrichment of the standard music notation with gestural and emotional annotations (Figure 16). By visualising this representation, the musical knowledge of both the composer and the performer is provided to the learner through i) the music score and ii) the gestural and emotional annotation of the expert performance, respectively (see D5.5, Section 4.4.3.2).



Figure 16: Graphical interface of the augmented musical score in the CMC game.

3.3.3 ICH indexing by stylistic factors and locality variations

Novel tools and techniques for stylistic analysis and comparison of ICH performances have been proposed based on statistic factor analysis and motion feature exploration. The aim is two-fold: i) to study performances of the same ICH and explore the effect of different inter-individual variables (e.g., gender, performer, affective state, etc.) on the realisation of the same high-level pattern; ii) to study performances of different ICH expressions of the same category (e.g., two different types of dance, like Tsamiko and Walloon) to extract locality variations (see D4.5 “ICH indexing by stylistic factors and locality variations”).

More specifically, a cross-platform API (C++ library called MotionMachine) that enables the rapid prototyping, extraction and visualisation of motion features has been developed and publicly released3 [55],[56] (see D4.5, Section 2). Several modules for feature extraction and stylistic analysis have been built based on this framework:

o A large set of stylistic feature extractors has been implemented in the MotionMachine environment. These features focus on different aspects of the movement, including kinematics, kinetics, ergonomics, and different concepts of Laban dance movement analysis. In addition to these high-level features, gesture recognition algorithms such as HMM and DTW have been also imported as full-part features (see D4.5, Section 2.2).

o A statistical exploration module has been developed [57],[58], which allows statistical analysis and comparison of different inter-individual (e.g., social, psychological, physiological and psycho-physiological factors, like gender, age, size, etc.) and intra-individual variables (factors related to the performance itself, such as type of the dance) and their effect on the extracted motion features (Figure 17) (see D4.5, Section 2.3). This module has been used for

3 MotionMachine toolkit: https://github.com/numediart/MotionMachine



• Factorial analysis of improvised contemporary dance to analyze the effect of different motion features (energy, distance, area, load, expansion and deviation) on six defined factors: gender, dance, dancer, arousal, valence and improvisation [57] (see D4.5, Section 2.4.1).

• Stylistic comparison (expertise level) of Walloon dancers, where HMM models have been used to evaluate a Walloon dance beginner against a professional dancer by providing a score related to the performance. In order to obtain more information about a performance, e.g., study whether the student is moving faster than s/he should or if s/he is not crossing her/his feet sufficiently, we can use the statistical factors and feature exploration analysis module (see D4.5, Section 2.4.2).

• Stylistic comparison of Walloon dance vs. Tsamiko dance based on factorial analysis. The objective was to analyze both dances and compare them to one another in order to extract locality variations. The analysis revealed the characteristics of each dance and the factors that can distinguish them. For example, analyzing the “deviation” feature (which is related to the deviation of the body from the most balanced pose) allowed concluding that Tsamiko dance gestures are more balanced than Walloon gestures. By analyzing the “weight effort” (maximum of kinetic energy over a defined time interval), we concluded that both of them are not very energetic (see D4.5, Section 2.4.3).

Figure 17: Screenshots from software tools for stylistic analysis and statistical

exploration of dance performances. The tools were developed based on the MotionMachine Toolbox.

The MotionMachine toolbox and related tools have been tested not only in the context of the dance use case, e.g., for extracting features and exploring their usefulness for the evaluation of dance performances/performers, but also in the context of other types of physical activity such as the Tai-chi-chuan Chinese martial art [58]. Moreover, this toolbox has also been used by contemporary artists to design and create artistic installations based on dance analysis results.

Finally, we have also developed a MyoWebToolkit for the purpose of recognizing and understanding finger gestures in the pottery use case (Figure 18). The MyoWebToolkit4 is capable of both annotating and recording the electromyography (EMG) signals that are captured by the MYO sensor during the execution of a certain gesture, and using these signals to drive the 3D model of a fore-arm. Through

4 MyoWebToolkit: http://augreal.mklab.iti.gr/MyoWebToolkit/ (use only with Chrome)



the use of this toolkit, we have been able to study the fine stylistic differences of a potter when crafting different types of objects [59],[60] (see D4.5, Section 3).

Figure 18: MyoWebToolkit simulation tool.

3.4 WP5: The Integrated Platform for Research and Education The main results of the research performed within WP5 are: a) a novel methodology for ICH education, b) a tool for the design of innovative educational interventions for ICH learning, c) a set of pedagogical plans for the ten different ICH sub-use cases created using the aforementioned tool, d) a set of LMS courses including several educational activities for learning the basics of ICH, which were developed based on the above mentioned pedagogical plans, e) a 3D visualisation platform for sensorimotor learning including game-like applications for practicing different ICH expressions, based on capturing, visualisation, analysis and evaluation of learners’ performances, f) novel algorithms for tongue modelling and visualisation based on ultrasound images, g) a customisable generic game, which provides an easy way to design and develop simple game-like applications for dance learning, h) a Text-to-Song synthesis engine with a Web interface for the automatic synthesis of traditional singing voices based on lyrics and notes, and i) an integrated Web-platform for ICH research, education, and transmission, which integrates all the aforementioned components.

More details can be found in the WP5 deliverables: D5.1 “Report on analysis of educational scenarios”, D5.2 “First version of 3D visualization for sensorimotor learning”, D5.3 “First version of Text to Song synthesis module”, D5.4 “First version of the Integrated Platform”, D5.5 “Final version of 3D visualization for sensorimotor learning”, D5.6 “Final version of Text to Song synthesis module”, D5.7 “Final version of the Integrated Platform”, and D5.8 “Final report on analysis of educational scenarios”.

3.4.1 Educational scenarios and tools

In order to support the creation of innovative educational interventions, especially for those ICHs where traditionally no formal teaching/learning practices exist, a novel methodology for ICH education was developed, which is based on three main steps: i) learning the ‘basics’, i.e., the corpus of theoretical knowledge that will allow



the learner to appreciate and understand the ICH in its intrinsic and cultural value; ii) exploring the ICH, i.e., being exposed to a number of expert performances and learning to appreciate/detect the main features of the ICH itself and the different styles/variations; iii) immersing in the ICH, i.e., having the chance to perform and having the possibility to get feedback in such a way, so as to detect one’s mistakes and improve the quality of the performance. This methodology has been the basis for the development of all the educational scenarios conceived and delivered within i-Treasures (see D5.8, Section 4) [61],[62].

Exemplar “pedagogical plans”, inspired by this methodology, were prepared in collaboration with the various ICH experts for the different ICH sub-use cases. These plans were planned thanks to an ad-hoc tool called the Pedagogical Planner (PP)5, which was especially conceived to support learning design in the field of ICH [63],[64],[65]. The plans cover different learning situations and different target populations, aiming to extend the educational opportunities to situations and segments of population that are not usually involved/interested in (Figure 19). Most of the exemplar plans have also been implemented into courses, which are delivered to users through the Learning Management System of the i-Treasures Web-platform6. The pedagogical plans, together with the related implemented courses, constitute an entity that in i-Treasures is called “educational scenarios”. To be noted that the development of the PP, which is able to support learning design in all its phases (from the initial conceptualisation of plans, down to their planning and implementation into actual courses on the LMS) is per se extremely innovative for the Technology Enhanced Learning (TEL) research field and this has been acknowledged with a “Best Paper” prize awarded to the paper describing this work at an EDEN conference [64] (see D5.1).

Section 1

General information

Section 2

Key aspects Section 3

Flow of activities

Figure 19: Main sections of the i-Treasures pedagogical plans.

5 i-Treasures Pedagogical Planner: http://i-treasures.itd.cnr.it/index.php 6 i-Treasures LMS: http://i-treasures.multimedia.uom.gr/drupalprivate/lms



In total, eight courses have been implemented from the corresponding pedagogical plans addressing the following ICH expressions: Tsamiko dance, Salsa dance, Canto a Tenore, Cantu a Paghjella, Byzantine music, Human Beat-box, wheel-throwing pottery and contemporary music (see D5.8). These courses have been developed in the local language of ICH practitioners/learners (Greek, French or Italian) but are also available in English (Figure 20).

Overall, the educational scenarios (pedagogical plans + related courses) aim to demonstrate the tools and affordances available in the i-Treasures platform in different contexts (schools, museums, etc.) and learning situations (formal and informal). The courses allow to demonstrate the platform in blended and distant learning initiatives, while offering a wide variety of possible activities, including self-study activities based on documents or audio/video materials, discussion and sharing activities, activities based on the game-like applications for sensorimotor learning, the Text-to-Song module and so on. Moreover, each course is enriched with exercises and tests, in order to make the learning experience more personalised and to give both the learner and the teacher a clear feedback about the learner’s achievements. In addition, useful analytics (including analytics already powered by the LMS, as well as analytics extracted by the game-like applications reflecting user behaviour and game usage) can be effectively visualised to allow the teacher to assess the performance of individual students or the class as a whole.

Figure 20: The LMS of the i-Treasures Web-platform. An example of a course on traditional wheel-throwing pottery.

3.4.2 3D visualisation module for sensorimotor learning

In the same direction, a novel 3D visualisation module for sensorimotor learning has been released, aiming to provide support for learning different types of ICH expressions [66],[67],[68],[69],[70]. The sensorimotor learning module was endowed with game-like applications, following a well consolidated trend in the TEL field that promotes the adoption of digital games to sustain learning and training in a variety of educational fields, aiming to empower constructive, experiential, self-regulated learning and increase the user’s engagement and motivation. Seven prototype educational multi-platform game-like applications have been implemented for selected ICH sub-use cases (Tsamiko dance [71],[69], Walloon dance [25], Calus



dance, Human Beat Box singing [72], Byzantine music [73], pottery, and contemporary music composition [74],[75])7. The games are presented in the framework of a single application and are designed to receive input from various sensors and game devices, such as the prototype hyper-helmet or off-the-shelf commercial sensors like Kinect (see D5.2 and D5.5).

Figure 21: Screenshot from the game-like application for Tsamiko dance. It presents the ghost avatar implementation, which allows the user to see his/her own virtual character ‘superposed’ with the expert’s one so as to be able to visually detect when s/he is making a mistake.

The basic game scenario involves a user, herein after called the learner, who learns the basics of the corresponding ICH by observing (watching and/or listening) and imitating expert performances. Expert and learner performances are visualised by 3D virtual characters in interactive 3D worlds, integrating multiple elements of the specific ICH tradition (e.g., traditional costumes, artifacts, symbols, locations). Depending on the game, additional informative views are also offered, e.g., video synchronised with the 3D avatar, close-up camera focusing on specific body parts (e.g., hands in pottery or feet in dance), or windows displaying different sensor signals (e.g., ultrasound video or 3D tongue model in singing). The learner’s performance is captured by the various sensors, it is analysed by the modules developed within WP3 and is, finally, evaluated by comparison with expert performances; the evaluation result is presented to the learner using audio-visual feedback (Figure 21, Figure 22). An independent component collects data during the use of the game (time spent in each activity, evaluation scores, etc.) and transmits these game analytics to the i-Treasures Web-platform (see D5.5, Section 6).

In addition, we have also developed an offline game-like application (the Pottery Master), in which the player can create a virtual pot by hand/finger gestures captured by a Leap Motion sensor. The shape and size of the pot changes dynamically as the player interacts with the virtual clay trying to create an object that matches the one she/he selected from an available list. The resulting 3D object can be extracted in a format that can be processed by 3D printers (see D5.7).

7 i-Treasures game-like applications: http://i-treasures.multimedia.uom.gr/drupalprivate/node/36



(a) (b)

(c) (d)

(e) (f)

Figure 22: Game-like applications for ICH learning & practicing: a) Virtual music studio environment in HBB game; the virtual tutor, which helps the user throughout the game, can be seen on the bottom left. Additional views on the left include video focusing on lips movement, tongue ultrasound and 3D model simulating tongue movement, b) Church environment in Byzantine music game; the GUI includes a metronome for keeping the rhythm and video of Byzantine music scores (i.e., the sequence of neumes), c) Virtual dance hall environment in Calus dance game; the expert avatar performs the basic dance movements; additional views zoom on the feet. d) Belgian countryside environment in the Walloon dance game. e) Virtual pottery workshop environment replicating the underground pottery workshops seen in Avanos, Cappadocia, Turkey. f) Music room environment in the contemporary music composition game.

The development of the games presented significant challenges, since it required: a) the integration of various sensors, software modules, operating systems, third party controller tools, network protocols, into a common framework; b) the development of novel 3D visualisation techniques, e.g., tongue movement simulation and visualisation based on ultrasound images; and c) the implementation of different game scenarios, taking in mind the specificities of each ICH sub-use case and the



modalities involved, which, in turn, affect several aspects of game design from visualisation and game assets to user feedback [68],[69],[70]. Despite the challenges, a robust common framework was implemented, which ensures easy development of future games for other ICH use cases and seamless integration of new software modules and sensors.

We have also developed a generic framework, which provides an easy way to design and develop simple game-like applications for dance or other kind of activities involving human motion [76] (see D5.5, Section 5). This framework consists of two components: i) an interface that allows the user to design a game scenario comprising of a set of sequential lessons/activities, provide descriptions for these activities, load/capture the corresponding expert performances using different motion sensors and video cameras, annotate the captured data, define how the evaluation of the learner’s performances will be performed, and select the avatar and the 3D environment. All this information is saved in an XML file; and ii) a generic customisable game for dance learning and training using off-the-shelf motion capture sensors, like Kinect, which can be automatically configured, based on the XML output of the previous module. Using this framework, game-like applications similar to those developed for Tsamiko/Walloon/Calus dances can be easily created. The exploitation possibilities of this tool are numerous, not only in the dance domain (a game for the popular Salsa dance has already been released) but also in other domains such as physical exercise, martial arts, physiotherapy, rehabilitation, etc. This game supports capturing, visualisation and evaluation of the user’s performance, without depending on other applications (like the WP3 modules) for data acquisition and user evaluation. Using this tool, we have already developed a suite of 11 game-like applications for learning Greek traditional dances, created in cooperation with dance experts.

Short demonstration videos for these games can be found in the project Website8.

3.4.3 Text-to-Song synthesis module

Another interesting educational tool is the Text-to-Song (TTS) synthesis engine for Sardinian Canto a Tenore, Byzantine music and English pop, which enables the user to enter text and/or notes and produce the equivalent singing voice (see D5.3 and D5.6). This module has been implemented as a Web service in the i-Treasures Web-platform9, and, currently, allows users to enter or select lyrics and produce Canto a Tenore songs, as well as enter lyrics and notes (or neumes) and produce Byzantine hymns and English pop songs (Figure 23). In the future, other forms of traditional or contemporary singing could also be supported. This module builds on novel natural language processing and voice synthesis algorithms and the recording of large singing voice corpora for the selected languages [77]. An API for handling new languages, as well as pitch and phoneme duration, has been developed and is made available for non-commercial and research use10. Moreover, the recorded Byzantine music and Canto a Tenore singing voice corpora were also released, in order to help other researchers in their efforts to study the rare singing cases that we have focused on. The corpora contain phonetically and musically rich recordings for the studied singing styles, along with their phonetic and musical segmentations11.

8 Videos for i-Treasures games: http://www.i-treasures.eu/demos 9 Text-to-Song interface in i-Treasures Web-platform: http://i-treasures.multimedia.uom.gr/drupalprivate/it_tts 10 API for singing voice synthesis: http://www.i-treasures.eu/content/softwares 11 Singing voice corpora: http://www.i-treasures.eu/content/datasets



The TTS synthesis system is based on a novel unit selection algorithm that uses the longer duration of phonemes during singing to smooth the spectral discontinuities that might occur between adjacent segments. The quality of the system was compared to contemporary algorithms developed by other research teams during the “Singing Synthesis Special Session: Fill-in the Gap” at INTERSPEECH 2016 Conference [77]. The results showed that our system is similar in quality to the best teams in the field.

Figure 23: Text-to-Song interface in the i-Treasures Web-platform: a) selection of type of voice (soloist or one of three accompanying cantors) and lyrics for the synthesis of Canto a Tenore songs (left), and b) selection of lyrics and neumes (type of musical annotation) for the synthesis of Byzantine hymns (right).

3.4.4 Web platform for research and education

The aforementioned tools and modules developed within WP3, WP4 and WP5 have been integrated into an open and extendable platform for ICH research and education. The i-Treasures Web-platform (http://i-treasures.multimedia.uom.gr/) is implemented as an open-source content management system enriched with a significant number of functionalities contributing to the preservation and widespread diffusion of ICH treasures [78],[79] (Figure 24(a)). It allows users not only to learn about important aspects of different types of ICH via the use of sound educational courses in different languages accessed through the LMS, but also to practice different types of ICH via the use of engaging educational games for sensorimotor learning, the novel Text-to-Song synthesis module and the innovative intangible musical instrument. The LMS environment has been modified to support visualisation of learning analytics collected not only from the use of the LMS courses (standard statistics supported by the LMS) but also from game usage. This way, it can effectively summarise the progress of each student or the classroom as a whole.

Moreover, it includes a digital repository, where ICH expert performances and corresponding high- and medium-level metadata can be stored or accessed. The user can search the digital repository for stored ICH content using advanced search



functionalities12 that exploit semantic analysis information (Figure 24(b)). New expert performances and extracted medium level features can be uploaded through the Web-platform interface, while corresponding high-level metadata information is automatically generated by the semantic analysis module. Metadata information is compatible with the Europeana Semantic Element Set [80]. The platform offers a user-friendly interface and multilingual support (see D5.7).

Figure 24: The i-Treasures Web-platform for research and education: a) use cases overview (left), and b) advanced search in the digital ICH repository using semantic analysis metadata (right).

3.5 WP6: System Demonstration In WP6 “System Demonstration”, a set of implementation and operational plans for the demonstration of the i-Treasures platform in the context of the ICH sub-use cases have been developed mainly based on the educational scenarios created within WP5 (see D5.1). The demonstration plans outlined the scope of the demonstrations and the scenarios to be followed, the type of users and experts to be involved, the schedule for the establishment and operation of the demonstrators, and the methodological approach for data/information collection (see D6.1 “Demonstration plan”).

The demonstration activities organised during the final year of the project involved a variety of users, covering a wide range of population including students of all ages, experienced ICH practitioners, ICH apprentices, researchers and general public in different European cities and villages. The activities also covered different scenarios ranging from structured learning paths for school students, to online courses for ICH beginners, to applications for museum visitors, to demonstration sessions for the general public during festivals, to the use of our technologies for the creation of new educational material or analysis of new ICH expressions. The demonstration events were organised in a variety of premises, including public schools, universities, dance schools, pottery workshops, museums, festivals as well

12Advanced search in i-Treasures ICH Repository: http://i-treasures.multimedia.uom.gr/drupalprivate/repository_search



as open days and dedicated workshops organised by the Consortium. In total, 26 different activities were organised in the context of 10 sub-use cases, which involved more than 650 users of all ages and with different backgrounds (see D6.2 “Demonstration report”).

User feedback was collected by means of questionnaires and interviews/discussions. Questionnaires were used for use case eevaluation in WP7. Moreover, valuable data, in the form of learning analytics, was collected via the use of the LMS and the game-like applications.

3.5.1 Demonstration for the traditional singing use case use

The i-Treasures technologies for rare singing capture, analysis and education were demonstrated in the context of the four singing sub-use cases. The demonstration scenarios varied from structured educational paths for school students to online workshops/courses for experts or the general public to demonstration of the technologies in science festivals (see D6.2, Section 3.1). The main technologies showcased were the hyper-helmet and the i-THRec and i-THAn tools for vocal tract capture and analysis, the LMS courses and the educational game-like applications developed for the different types of singing, the TTS module and the i-Treasures Web-platform as a whole. In total, 11 activities took place and more than 250 users were involved, as summarised below:

• Canto a Tenore

Structured educational path for secondary school students: 10 students of a secondary school in Sardinia followed the learning activities of the LMS course for Canto a Tenore as an extra-curricular school activity. The course was taught by a Canto a Tenore expert and its various activities were alternated with real singing sessions, during which students could practice the Canto with the help of their teacher.

Online workshop for music experts: a group of 10 experts (music teachers, singers, ethno-musicologists) participated in an online workshop that aimed at collecting their opinion about the potential of the i-Treasures platform and LMS course for apprentices in Canto a Tenore and for student singers in general. After an initial presentation, the participants explored the Web-platform and followed the educational path of the LMS course by themselves, being assisted by a Canto a Tenore expert when they needed help.

Open online course on the basics for the general public: a permanent online course for anyone interested to learn the basics of Canto a Tenore. The course was followed by 39 people (Canto a Tenore singers or apprentices, music teachers, singers, people interested in Sardinian culture). The participants could communicate with experts through dedicated LMS forums. To the best of our knowledge, this is the first course on the Canto a Tenore singing tradition and, as such, it is an important legacy of the project.

• Cantu in Paghjella

Workshop in museum: A workshop dedicated to i-Treasures technologies for Cantu in Paghjella safeguarding was organised in the museum of Corte in Corsica and the audience was able to use the technologies during a “hands-on session”. The participants (40 local people and 10 Paghjella experts) working in groups followed the learning activities of the LMS course and were also able to test the hyper-helmet. In addition, live singing activities were organised during the breaks, allowing the attendees to practice with groups of Cantu in Paghjella singers. The LMS course will serve as a



promotional/educational tool aiming to introduce Cantu in Paghjella to museum visitors.

Online course on Cantu in Paghjella basics: The learning activities of the course were followed by a group of university students with an interest in music at the CNRS lab in Paris.

• Byzantine music

Structured educational path for primary school students: 20 students of the primary experimental school of the Aristotle University of Thessaloniki followed the learning activities of the LMS course for Byzantine music in the context of their music class. The course was taught by a music teacher/Byzantine music expert and included activities with the corresponding game-like application and TTS module as well as live singing sessions.

Educational tools for Byzantine music chanters: The activity took place in a church in Larissa, Greece and involved a group of 5 amateur Byzantine music singers that had basic knowledge of this ICH and were practicing on their own. The demonstration covered the theoretical part that has been implemented in the available LMS course and also included a practice session based on the use of the game-like application.

Structured educational path for adults: a group of university students with established musical education but no background in Byzantine music followed the LMS course activities in the premises of UOM, practiced with the game and explored the TTS tool.

• Human Beat Box

Introduction to HBB for children: An alternative to discovering HBB in a non-formal way, suitable for young children, was presented during the French Sciences Festival in Paris. It involved an HBB expert and 40 children attending the event with their music class and teachers. The hyper-helmet was used as an educational tool that helped the expert to illustrate what happens during the production of a singing voice. The different activities of the LMS course were presented based on a playful approach actively involving the children, while a practical session allowed young learners to practice different HBB sounds.

Introduction to HBB for the general public in a festival: An example about the kind of reach-out or informational activities that could be easily organised in music events/festivals (or even museums) for increasing the general public’s awareness and interest about a singing tradition based on the tools developed within i-Treasures was presented during the French Sciences Festival in Paris and involved a group of 60 adults. The HBB expert presented the hyper-helmet and different activities of the LMS course and participants tested them.

Online course on HBB basics: The learning activities of the course were followed by a group of university students with an interest in music and hip-hop culture at the CNRS lab in Paris.



Figure 25: Demonstration activities in the context of the singing use case: a) Canto a Tenore course for students of a secondary school in Sardinia, b) HBB demonstration to elementary school students and their music teacher in Paris, c) Workshop on “ICT technologies for Cantu in Paghjella singing and education” at the museum of Corte, Corsica, d) Byzantine music course for elementary school pupils in Thessaloniki, e) HBB demonstration for the general public during a science festival in Paris, and f) Online demonstration of Canto a Tenore LMS course to a group of music experts from Italy.

3.5.2 Demonstration for the dance use case

The i-Treasures technologies for rare dance capture, analysis and education were demonstrated in the context of the following sub-use cases: Tsamiko dance, Salsa dance, and Walloon dance. The demonstration scenarios vary from structured educational paths for students to online courses for amateurs to demonstration of the developed technologies in dedicated workshops (Figure 26).

The main technologies showcased were the various modules for dance capturing and analysis developed in WP3, the MotionMachine toolbox for stylistic analysis, the Pedagogical Planner tool, the LMS courses and the educational game-like applications developed for the aforementioned types of dancing, and the generic dance game development framework. In total, 8 activities were organised and approximately 140 users were involved (see D6.2, Section 3.2).

• Tsamiko dance

Structured educational path for university students: 33 students attending the “Didactics of Greek Traditional Dances” course in the Department of Physical Education and Sports Science of the Aristotle University of Thessaloniki followed the learning activities of the LMS course for Tsamiko dance. Theoretical activities were followed online at home, while practical activities with the corresponding game-like application and real dance sessions took place in the dance hall of the School.

Workshop on ICT technologies for dance education: a workshop addressed to dance stakeholders but also to ICT researchers with research interests focused on motion analysis was organised in Thessaloniki. The different i-Treasures technologies for dance capture, analysis and education were presented to the audience, including a live demonstration of the generic game-development framework and the game-like applications for different traditional dances. A discussion about the advantages and weaknesses of our



tools and ideas for possible extensions in other areas involving physical exercise and human motion followed.

• Walloon dance

Workshop on traditional dance and new technologies: a workshop for showcasing the use of i-Treasures tools for the preservation of Walloon traditional dances and raising awareness about this endangered tradition to the general public and the scientific community was organised in the Numediart Institute in Mons. The Walloon dance game-like application was demonstrated and the participants (dancers, university students and general public) were able to learn and practice the basic steps of the dance.

Use of educational tools for Walloon dance learning in traditional dance festival: the i-Treasures tools for dance capture and analysis and the game-like application for Walloon dance were presented to the general public in the context of a local event about traditional Walloon dances and songs at the cultural centre of Beloeil in Belgium. The participants had the opportunity to use the game and learn the basic Walloon dance steps.

• Salsa dance

Structured educational path for dance school students: 8 beginners attending a Salsa class in a local dance school in Thessaloniki and 3 dance teachers followed the learning activities of the LMS course for Salsa. Theoretical activities were followed online at home, while practice activities with the game-like application and live dance sessions took place in the dance school.

Open online Salsa course for amateur dancers: the idea was to offer an online course for anyone interested in learning the basics of Salsa. 9 people expressed interest and participated. Most of them were practicing other types of dance as amateurs and were interested in getting familiar with new dances and/or were curious to experience non-traditional ways of dance learning. After completing the theoretical activities of the LMS course at home, the learners were invited at CERTH premises to complete the activities that aimed at fostering their dancing competency through the game-like application and face to face live dance sessions.

• Tai-chi-chuan martial art

Workshop on the use of i-Treasures tools for Tai-chi-chuan: A workshop was organised at the Numediart Institute in UMONS aiming at demonstrating the usability of the MotionMachine toolbox in areas beyond dance, and more specifically, in the case of the Tai-chi-chuan martial art. The first phase included a recording session that involved capturing the participants while practicing Tai-chi-chuan. A new motion capture dataset for Tai-chi-chuan was collected, including 12 subjects and approximately 1500 motion sequences. On a follow-up event, MotionMachine and other related tools were used by researchers and Tai-chi-chuan experts to analyze the captured data, explore high-level features, develop new features related to Tai-chi-chuan (defined by the experts), and compare the automatic evaluation of the Tai-chi practitioner’s performance (using our tools) with the evaluation made by the experts.



Figure 26: Demonstration activities in the context of the dance use case: a) Students of the Physical Education and Sports Science Department of the Aristotle University of Thessaloniki engaging with the Tsamiko game-like application, b) Workshop on “ICT technologies for dance research and education” in Thessaloniki, c) Demonstration of game-like application for Salsa in a dance school in Thessaloniki, d) Demonstration of Walloon dance game-like application during a folk dance event in Belgium, e) Demonstration of motion capture and stylistic analysis tools to Tai-chi-chuan learners and experts at the University of Mons.

Most of the demonstration activities organised for the dance use case as well as for the other ICH use cases were based on scenarios where a group of learners used the educational material and tools developed within the project to learn the basics of an ICH and practice it. In the context of the dance use case, we decided to also showcase the usability of the i-Treasures platform in a scenario where the developed tools are used by experts to create their own educational material. This activity took place at the Department of Physical Education and Sports Science of the Aristotle University of Thessaloniki and involved 13 undergraduate students, whose major/specialty was “Dance didactics”, and the supervising Professor. Using the Pedagogical Planner tool and the generic dance game development framework, the students designed pedagogical plans for 11 traditional dances from Greece and Cyprus and developed 11 corresponding game-like applications. Moreover, LMS courses were also developed for 4 of these dances (Figure 27).

Figure 27: Screenshots from the educational tools for learning Greek traditional dances, developed by students of the Physical Education and Sports Science Department of AUTH: a) LMS course for the traditional dance Sybethera from the island



of Ikaria (left), b) observe mode of the game-like application for Leventikos dance (right).

3.5.3 Demonstration for the pottery use case use

The demonstration scenario for the pottery use case involved a structured educational path for adults that had no prior knowledge of pottery but wanted to acquire basic knowledge and pottery making skills. It was realised in the context of a pottery class taught by an expert potter in a municipal cultural centre in Thessaloniki and involved a group of 6 learners, all of them beginners. The focus of the demonstration was the LMS course for pottery as well as the Pottery Master game-like application. The learners followed the theoretical activities of the course online at home, while the practice activity with the game-like application as well as the session that involved real practice with clay took place in the pottery workshop (Figure 28) (see D6.2, Section 3.3).

Figure 28: Demonstration for pottery use case in a pottery workshop in Thessaloniki.

3.5.4 Demonstration for the contemporary music composition use case use

Three different scenarios were envisaged for the demonstration of the contemporary music composition use case, which resulted in 6 activities that involved more than 250 participants (Figure 29) (see D6.2, Section 3.4). The main technologies showcased were the LMS course for contemporary music, the corresponding game-like application and the intangible musical instrument:

• Structured educational path for adults interested in music composition: 33 university students and musicians followed the learning activities of the LMS course, which also included use of the game-like application for learning expert musical gestures and embracing specific emotional states by observing different affective stimuli.

• Use of the IMI for learning musical gestures: Two demonstration activities were held in the context of international conferences in Thessaloniki and Sydney and another one was held in an incubator in Paris. They involved 70 university students, musicians, researchers and the general public. Participants used the IMI for a) learning simple musical gestures based on imaginative gesture sonification, and b) composing novel pieces of contemporary music by combining different kinds of musical gestures and mapping them to different sounds.

• Composition and performance of musical pieces based on the IMI: A new contemporary music piece, called IDEASMOS, was composed by the music composer Ms Vasiliki Tsekouropoulou by combining acoustic musical instruments (two violins, a viola and a violoncello) with the IMI (two IMIs). Two live concerts have been organised: one at the Concert Hall of Thessaloniki and another at MINES ParisTech in Paris during the “Musique aux Mines” festival.



Figure 29: Demonstration activities for the contemporary music composition use case: a) adults following a structured educational path for music learning in the i-Treasures LMS (left), b) participants experimenting with the IMI in the context of the music performing/composing scenario (middle), and c) music concert of IDEASMOS (with IMIs and acoustical musical instruments) in Paris (right).

3.5.5 Demonstration challenges and conclusions

The design and organisation of these events presented significant challenges, which included handling of different groups of users with various educational, professional or national backgrounds and different competency levels, reluctance of some users to participate and negativity/fear towards new technologies, technical difficulties, etc.

Despite the numerous challenges, the demonstration of the i-Treasures Web-platform has been very successful. The users were, in general, satisfied by the developed technologies and thought that they constitute an innovative and promising set of tools that can help and advance ICH analysis, education and research. They felt that the offered tools respect ICH tradition and can significantly contribute to its safeguarding and transmission, taking advantage of recent advances in ICT technologies to analyse ICH content in a deeper and more effective way and to boost ICH education in ways that make it more appealing and attractive to younger generations.

The experts have played a crucial role in introducing and explaining the potential of our technologies to local people and convincing them to use them: first, by highlighting the aspect of capture and analysis, which allows for the better understanding and documentation of ICH, and secondly, by underlining the advantages of modern tools for ICH education that are more appealing to young people and also attract possible interest from around the world, making their culture known to and appreciated by a much wider audience, which would not be reached otherwise. If the experts incorporate new technologies in their everyday practice, then they will much more easily convince other practitioners to accept and use them as well.

3.6 WP7: Technical Assessment and Evaluation In WP7, a detailed assessment plan and evaluation methodology has been developed, which focuses on two perspectives, i.e., performance assessment of the developed technologies and usability evaluation of the system [7] (see D7.1 “Assessment plan”). The assessment process is performed in three ways: a) individual WP3, WP4 and WP5 modules were tested in laboratory conditions by the consortium, b) the integrated system was evaluated in the context of the various sub-use cases in real conditions with real users that provided feedback via questionnaires, c) technical assessment of the i-Treasures system was performed



with the help of IT experts who also filled in questionnaires (see D7.3 “Final evaluation report”).

3.6.1 Laboratory testing

The technical performance of the i-Treasures modules has been assessed in laboratory conditions through a set of tests (see D7.3, Section 4). Towards this direction, a tool based on fuzzy logic (FL), namely fisEval, has been developed for fusing (in a smart way that incorporates the expert knowledge) the divergent assessment indices of each module (e.g., accuracy and speed), and deriving an overall performance index (PI) that characterises the module (Figure 30). The technical assessment results show that the final versions of the technological modalities developed are of satisfactory quality and cover, to a high extent, the functional and operational requirements set by the users. The overall performance indices acquired during the second phase of laboratory testing exceed those of the first phase, and all modalities surpass the baseline criteria that were defined by the developers according to similar state-of-the-art approaches and desired performance. Figure 31 presents the PIs of the individual modules that were tested in laboratory conditions.

Figure 30: Graphical user interface of the fisEval tool for the technical assessment of system modules.

Apart from fisEval, a new tool named iTreasuresEvaluator was developed, in order to facilitate the presentation of the assessment and evaluation results of the i-Treasures platform at any structural depth. More specifically, iTreasuresEvaluator implements the FL-based evaluation methodology proposed in the assessment plan and permits the user to calculate and retrieve the evaluation results of individual modules, sub-use cases, use cases, the entire platform and other entities, i.e., data fusion and semantic analysis, Web-platform, and generic game. With the aid of this tool and dedicated fuzzy inference systems, a technical performance index (TPI) has been calculated for each sub-use case. Figure 32 presents the TPIs and baseline TPIs for all sub-use cases. Based on these results it can be concluded that the performances of all sub use cases are rather solid and surpass the corresponding baseline indices by 13% at least.



Figure 31: Performance indices (PIs) of the individual modules for the two laboratory testing phases as calculated by the fuzzy logic-based fisEval tool.

Figure 32: Technical performance indices (TPI) of all sub use cases along with the corresponding baseline TPIs.

To conclude, the above assessment results denote that the modalities of the i-Treasures platform, considered as standalone working units, are of high quality, from a technological point of view, exhibiting rather satisfactory performance and covering the majority of user requirements and system specifications. Although they have received effective improvements over the course of development phases, there are certain limitations that should be addressed in the future. However, these limitations do not hinder the overall functionality and quality of the i-Treasures platform.

3.6.2 Use cases evaluation

In parallel, several activities have been organised for the evaluation of the i-Treasures platform in real-world settings with real users, based on meaningful case study scenarios and well-defined evaluation protocols (see D7.3, Section 7). The use cases evaluation was performed in two phases, following the technology development cycles of the project. During the first phase, tests for selected sub-use cases (Tsamiko dance, Human Beat Box singing, pottery and contemporary music composition) were organised in music schools, pottery workshops, dance halls, and university labs and valuable feedback was collected, which was used as a starting point for updating the user requirements and system specifications in the second phase of development. In the second phase of the evaluation, the evaluation was based on the output of the demonstration activities organised for the all the different sub-use cases (see above). Evaluation data in both phases were mainly collected through structured questionnaires filled by the participants. Analysis of evaluation



results was based on the innovative methodology of structural equation modelling [81], which allows the identification of the strengths and weaknesses of the integrated system and its parts. Additionally, the contribution of each component of the system towards the performance of other components, and, in turn, towards the overall performance of the system was found. The overall performance of i-Treasures, in a 5-level scale (i.e., ‘1’=very poor, ‘2’=poor, ‘3’=average, ‘4’=good, ‘5’=very good) was found to be equal to ‘3.95’ (‘good’) (see D7.3, Section 6). Moreover, the second-phase evaluation results denote that, in general, the participants were positive towards the technologies developed within the project and optimistic about the potential of the i-Treasures Web-platform for the preservation and transmission of ICH. They also pointed out some weaknesses of the developed technologies and proposed interesting extensions, which will be considered in future development efforts in order to augment the value of the i-Treasures platform.

Figure 33: Evaluation of i-Treasures technologies by: a) university students from the Dept. of Physical Education and Sport Science of AUTH in the context of the Tsamiko dance sub-use case (left), and b) music school students in the context of the HBB singing sub-use case (middle), and c) IT expert filling in the e-questionnaire for the technical assessment of system modules (right).

3.6.3 Technical assessment of the system

Finally, a procedure for the overall technical assessment the i-Treasures system and validation against the user requirements took place (see D7.3, Section 5). Two structured questionnaires, one for the overall assessment of the system based on non-functional requirements (e.g., accessibility, usability, extensibility, etc.) and one for the assessment of the main technical indices (main system components, i.e., human body analysis module, TTS module, educational platform, etc.) based on functional requirements, were prepared and they were filled in by experts with advanced knowledge in informatics and, also, expertise in the research topics studied within i-Treasures [7][4]. The answers were in the Likert- scale (1 to 5).

The analysis of these assessment results was based on descriptive statistics and confirmatory factor analysis [82]. In the two Tables below, the mean scores and standard deviations of the user evaluation are presented [3]. The analysis results denote that both the system and its main components have been assessed to be at ‘level 4’, indicating that they are ‘good’. This analysis allowed us to indicate, which the strong components of the system are and where there is room for improvement.



Table 2: Indicative results of the technical system assessment based on non-functional requirements.

Table 3: Indicative results of the technical assessment of the main technical indices.

System components Mean score Std

Facial expression analysis 3.91 0.73

Human body motion and gesture recognition 4.01 0.60

EEG analysis 4.15 0.69

Vocal tract analysis 3.79 0.91

Sound processing 3.89 0.62

Text-to-song 3.95 0.70

Semantic Analysis 3.89 0.73

Web-platform 3.72 0.87

Non-functional requirements Mean score Std

Cost optimality 3.38 0.67

Accessibility / Usability 3.63 0.80

Documentation / Support 3.92 0.66

Interoperability / Portability 3.74 0.61

Extensibility / Scalability 3.68 0.69

Auditing 3.56 1.08

Security / Privacy 4.05 0.64

Fault tolerance / Recoverability 3.58 0.97

Licensing / Copyright 4.16 0.87



4. Potential impact, dissemination and exploitation

4.1 Impact The Intangible Cultural Heritage represents an important part of the world heritage. The i-Treasures project provides a novel strategic framework for the safeguarding and transmission of ICH, which is implemented through an open and extendable platform for research and education. The i-Treasures platform:

• Contributes to the safeguarding and documenting of ICH by developing

o Novel multi-sensor technologies for capturing different ICH expressions.

o Novel tools for the analysis of ICH elements based on body motion and hand/finger gesture recognition, facial expression analysis, vocal tract modelling, affective state recognition, and sound processing.

• Unveils unknown correlations and gives rise to a deeper understanding of specific forms of ICH by

o Novel methodologies for the extraction of new knowledge that enable researchers to identify possible implicit or hidden correlations between different ICH expressions or different interpretation styles of the same ICH and study the evolution of a specific ICH.

o Integration of heterogeneous ICH resources.

o Advanced semantic browsing and searching functionalities for ICH resources.

• Breaks new ground in education and knowledge transfer of ICH by

o Novel methodology for ICH education based on a 3-step approach: basics, exploration, immersion.

o Innovative solutions to support learning design and pedagogical planning in the field of ICH education.

o Novel educational scenarios (pedagogical plans + related courses) for ICH teaching and learning that can be delivered online, through a LMS, which also supports the monitoring of learners’ performance.

o Novel 3D visualisation platform for sensorimotor learning for the transmission of rare know-how from ICH experts to learners through serious game-like applications that allow learning/ practicing different kinds of ICH expressions.

o Novel singing voice synthesis tool for traditional singing.

• Creates new ways for cultural expressions:

o Novel intangible musical instrument for contemporary music composition based on natural gesture recognition and affective status surfaced by brain activity analysis.

• Provides an elegant, robust and easily extendable solution for ICH management and dissemination:

o Modular and flexible architecture of the Web-platform that allows for easy system upgrades and extensions depending on the particular needs of different ICHs.



o Open-source content management system enriched with a significant number of functionalities contributing to the preservation and widespread dissemination of ICH treasures.

The i-Treasures project already provides significant societal and economic benefits, including:

• Raising awareness of the importance of ICH and contributing to its safeguarding.

• Promoting specific forms of ICH (traditional dances, songs, handicrafts) that can be seen as a community treasure and enforcing the local development by making local ICHs more popular and dominant.

• Contributing to the development of a systemic methodology for the preservation, renewal and transmission of rare knowledge to next generations through the concept of training workshops. Approximately 100 individuals have been trained in local training workshops organised by the Consortium and many of them have already used the i-Treasures innovative technologies in combination with traditional safeguarding techniques to design their own ICH safeguarding projects in cooperation with local communities and organisations.

• Allowing a wider range of people to use cultural resources in diverse real and virtual contexts and considerably altered ways to experience culture in more personalised and adaptive interactive settings:

o ICH cultural resources are easily accessible through the i-Treasures Web-platform, enabling the widest possible participation of communities, groups and individuals in the safeguarding, transmission and research of ICH.

o The i-Treasures platform allows multi-lingual and universal access to ICH content, provides a highly customisable environment according to distinct user roles (learner/teacher/researcher), and supports a wide range of educational scenarios for ICH education via a user-friendly LMS environment.

o Using the developed tools, ICH experts can easily design educational interventions for ICH learning, including formal, non-formal and informal learning scenarios, targeting different populations, and offering a variety of learning activities covering theoretical and practical aspects of ICH based on innovative tools like the game-like applications for sensorimotor-learning or the TTS module.

o The i-Treasures tools offer the possibility for online learning and practicing, thus, allowing users from all over the world to learn the basics of ICH expressions practiced by small local and, sometimes, not easily accessible communities.

• Providing affordable tools for releasing the economic potential of cultural heritage in digital form and for adding value to cultural content in educational and scientific context.

• Strengthening the European cultural identity with the active engagement of European universities, research institutes and industries in the preservation and protection of ICH and maintaining cultural diversity and encouraging intercultural dialogue and respect for other ways of life.



• Contributing to the protection of cultural heritage, the basic asset on which tourism is built. Tourism, closely related to cultural heritage, is the third largest socio-economic activity in the EU and generates over 5% of the European GDP.

4.2 Dissemination activities The dissemination activities carried out by the Consortium included a series of actions that provided third parties outside the consortium with information relevant to the project, aiming to increase interest from stakeholders and support the exploitability of the i-Treasures system. The dissemination activities of i-Treasures include the following:

• Design of project logo, brochure, posters;

• Establishment of Web-site and social media accounts;

• Release of dissemination videos;

• Press releases and TV interviews;

• Organisation of workshops, special sessions, and other events for the scientific community and the general public;

• Organisation of the i-Treasures Workshop;

• Participation in conferences, workshops, events for the scientific community and the general public;

• Clustering activities with other research projects, organisations, industrial partners;

• User group establishment;

• Organisation of live concerts for traditional and contemporary singing;

• Organisation of local training workshops for safeguarding ICH;

• Publications in international journals, books and conferences.

The main goal of these activities was to raise awareness about the safeguarding of ICH, disseminate the project’s results, train users and exploit the i-Treasures integrated system and tools. More information about our dissemination activities can be found in deliverables D8.1 “i-Treasures Web site”, D8.2 “i-Treasures brochure”, D8.3 “Plan for the use and dissemination of i-Treasures”, D8.4 “First report on dissemination activities”, D8.5 “Second report on dissemination activities”, D8.6 “Third report on dissemination activities” and D8.7 “Final report on dissemination activities”.

4.2.1 Project logo

The project logo presents the project scope and technology in the simplest way. It is a representation of a group of four persons (Living Human Treasures), one for each i-Treasures use case, in a very abstract form, symbolizing how fragile they are. Based on the same idea, a banner was also designed showing the four abstract silhouettes of a singer, a dancer, a potter and a music composer (Figure 34).



Figure 34: The i-Treasures logo (left) and banner (right).

4.2.2 Project website

The i-Treasures Website is a tool for presenting the project to the outside community and facilitating on-line collaboration between partners (Figure 35). The Website is reachable via the domain www.i-treasures.eu. It contains a public and a private part. The private part contains a file repository where all project documents are stored and is updated on a continuous basis by CERTH (see D8.1). Software, datasets and publications are available at the project Web-site:

• Datasets: http://www.i-treasures.eu/content/datasets

• Software: http://i-treasures.eu/content/softwares

• Publications: http://www.i-treasures.eu/content/publications-0

• Deliverables: http://www.i-treasures.eu/content/deliverables-0.

Figure 35: The i-Treasures Website (www.i-treasures.eu ).

4.2.3 Social media

A Facebook page has been created for i-Treasures, which is accessible at https://www.facebook.com/pages/I-Treasures/495950407134546.

The Twitter account of i-Treasures is @iTreasures_.



4.2.4 Brochure

The project brochure was used to support project presentation in different events / meetings and has also served as a communication tool for potential mailings to groups of users. Except from the English version13, the brochure was also translated and printed in Greek, French, Italian and Turkish. Apart from providing an overview of the project consortium and some contact information, the six-page brochure mainly focuses on the project motivation, goal and main objectives, offering also an overview of the overall methodology. The four use cases are also presented. Having both end-user-oriented as well as technical content, the brochure appeals to all groups identified as target audiences. An electronic version of the brochure is available for download at the project Website (Figure 36) (see D8.2).

Figure 36: External and internal pages of i-Treasures brochure.

4.2.5 Posters

Several posters have been created by partners and have been used at events such as conferences, exhibitions and international fairs. All posters have the same look and feel as the Website and the brochure, following a similar design approach (Figure 37).

Figure 37: Poster of i-Treasures keywords.

13 i-Treasures brochure: http://www.i-treasures.eu/sites/i-treasures.eu/files/dissem-i-treas/brochure/brochure_en_final_v2.pdf



4.2.6 i-Treasures dissemination videos

A public showcase video demonstrating the objectives and initial results of i-Treasures was created in the first year of the project. The main focus of this promotional video is to inform the broad public about the i-Treasures overall benefits with respect to the selected ICH use cases and, also, demonstrate the advantages of the proposed multi-sensor approach over existing technologies for ICH transmission and preservation. Two versions of this video (short and long) were delivered. The video is available to the general public through the homepage of the i-Treasures Website and YouTube (http://www.youtube.com/watch?v=Yyt-FPU_O3Y). It has been shown in exhibitions, conferences and other dissemination events.

In addition, 20 short demo videos have been created by partners and have been uploaded at the i-Treasures Website (http://i-treasures.eu/demos). The main objective was to demonstrate the application of i-Treasures technologies in capturing and analysing different ICH expressions and present the different tools developed within the project.

4.2.7 Press releases and TV Interviews

Numerous press articles, interviews and press releases have been released in European media and are available via the project website (http://www.i-treasures.eu/content/dissemination). Briefly, these are presented below:

• TV interviews in European media:

o Interview of Dr. Stelios Hadjidimitriou on AlphaTV, Greece (8/11/2014).

o Interview of ARMINES researchers about the IMI on AlphaTV, Greece (5/7/2016).

o Interview of UOM researchers about the IMI on TV100, Greece (12/9/2016).

o Short documentary about the i-Treasures project and the safeguarding of the Walloon dance on Notélé, Belgium (12/2016) (Video: http://www.notele.be/list13-le-jt-a-la-carte-media47423-les-pas-de-la-y-au-au-secours-des-danses-wallonnes.html).

o Short documentary about i-Treasures filmed by Euronews TV during the 12th project meeting in Thessaloniki, on 22-23 June 2016. The crew filmed procedures for capture and analysis of different ICH performances, use of the game-like applications by learners in real-settings as well as mini-interviews with members of the Consortium and ICH experts. The documentary aired as part of the Futuris show in August 2016 (Video: http://www.euronews.com/2016/08/06/european-digital-database-aims-to-keep-cultural-traditions-alive) (Figure 38).

• Articles in European newspapers (Figure 39):

o Interview of Dr Nikos Grammalidis in Tipos tis Kiriakis, Greece (24/3/2013).

o Article about the use of EEGs for contemporary music composition in Walliser Bote, Switzerland (21/9/2013).

o Article about the use of EEGs for contemporary music composition in Radio Rottu Oberwallis, Switzerland (6/11/2013).





o Article about i-Treasures and Canto a Tenore in il Secolo XIX, Italy (21/1/2014)

o Article about i-Treasures and Canto a Tenore in L’Unione Sarda, Italy (5/12/2014)

o Article about i-Treasures and Canto a Tenore in La nuova Sardegna, Italy (5/12/2014)

o Interview of Dr Nikos Grammalidis in Aggelioforos, Greece (24/5/2015)

o Article on “Understanding the mechanisms of a Paghjella via technology” in Corse-Matin, France (26/9/2016).

• Articles and interviews that were released in European news portals:

o http://www.politismos-thessaloniki.gr/2015/01/blog-post_14.html

o http://www.nooz.gr/article/otan-o-egkefalos-ginetai-mousiko-organo

o https://www.cnr.it/it/evento/15037/open-online-course-sul-tema-patrimoni-culturali-immateriali-il-canto-a-tenore

Figure 38: Screenshots from the documentary about i-Treasures on Euronews TV.

Figure 39: Articles about i-Treasures in European newspapers.



4.2.8 Organisation of live ICH performances

Nine live concerts have been organised in the context of i-Treasures (Figure 40). These are listed below:

• A Human Beat Box concert was performed by the HBB artist DAVOX on 7/11/2013 in the ICT 2013 Event in Vilnius, Lithuania (Video: http://www.i-treasures.eu/node/274).

• A concert of the “Common Brain” contemporary music work (based on live electronics and EEG analysis) by the band UMS 'n JIP was held on 8/11/2013 in the ICT 2013 Event in Vilnius, Lithuania (Video: http://www.i-treasures.eu/node/275).

• The world premiere of “Brainswarm” contemporary music work (based on live electronics and EEG analysis) was held at the Goethe Athens Institute on 9/12/2013 (Video: http://www.youtube.com/watch?v=f53eXf4Q0gI).

• Two other concerts of “Brainswarm” were organised in the context of a) the "Contemporary music without codes" event, which took place at the Megaron Concert Hall of Athens on 20/12/2016 (Video: https://www.youtube.com/watch?v=jB8jBtvoAB8), and b) the joint 40th International Computer Music Conference – 11th Sound and Music Computing Conference in Athens, Greece on 18/9/2014.

• A performance of “Geografia Seleste”, a contemporary composition for voice and live electronics based on EEG data, was given by the lyric singer Ms Aggeliki Kathariou in the context of the "1st Athens Science Festival" on 12/5/2014.

• A Canto a Tenore concert performing the “Tenore Montalbo" di Siniscola group was held in Genoa, Italy on 23/1/2014 (Video: http://www.youtube.com/watch?v=Hf23FHEI_AI).

• Two concerts of “IDEASMOS”, a contemporary music piece combining acoustic musical instruments (2 violins, 1 viola and 1 violoncello) and the intangible musical instrument (2 IMIs), were organised in the framework of a) the 3rd International Symposium on Movement & Computing at the Concert Hall of Thessaloniki on 5/7/2016 (Video: https://www.youtube.com/watch?v=0rvOEgojcVs) and b) the “Musique aux Mines” festival in Paris on 7/9/2016.



Figure 40: Live concerts organised by i-Treasures: a) Canto a Tenore performance of

the Tenore Montalbo di Siniscola group in Genoa, b) Human Beat Box performance by DAVOX in ICT 2013 Event in Vilnius, c) “Brainswarm” concert at Megaron-The Athens

Concert Hall, and d) “Ideasmos” concert at the Concert Hall of Thessaloniki.

4.2.9 Organisation of conferences, workshops, special sessions and other events for the scientific community

The Consortium has organised three workshops, one symposium and four special sessions on issues related to cultural heritage and ICT, addressed at the scientific community (Figure 41). More specifically:

• The 3rd International Symposium on Movement and Computing (MOCO’16)14 was organised in Thessaloniki, Greece, on 5-6/7/2016.

• A Workshop on “ICT for the Preservation and Transmission of Intangible Cultural Heritage”15 was organised in the context of the Euromed2014 conference in Lemessos, Cyprus on 6/11/2014.

• A Workshop on “Digitising Humanities: Methodological reflections and prospects for the promotion and diffusion of the tangible and intangible cultural heritages”16 was organised at the University of Cagliari, Italy, on 22-23/6/2015.

• A Workshop on “Knowledge and Skills in the Intangible Cultural Heritage”17, including invited talks by ICH experts, was organised in Paris, France on 11/6/2013.

• A special session on “ICT-based Analysis and Modeling of Intangible Cultural Heritage”18 was organised in the context of the VISAPP2014 conference in Lisbon, Portugal on 6/1/2014.

• A special session on “Multimodal Capture, Modeling and Semantic Interpretation for Event Analysis, Retrieval and 3D Visualization” (MMS-ER3D)19 was co-organised in the context of VISAPP2015 conference in Berlin, Germany on 12/3/2015.

14 https://moco16.movementcomputing.org/ 15 http://www.culturalheritage2014.eu/files/4913/8864/9075/EuroMed2014_Call_for_Papers_Workshop_Intangible_Heritage.pdf 16 http://www.isem.cnr.it/ProgHumanities.pdf 17 http://www.i-treasures.eu/sites/i-treasures.eu/files/Workshop_Programme.pdf 18 http://www.visigrapp.org/IAMICH.aspx?y=2014 19 http://www.visigrapp.org/MMS-ER3D.aspx?y=2015



• A special session on “RGB-SpectralImaging”20 was co-organised in the context of the VISAPP2016 conference in Rome, Italy on 27/2/2016.

• A special session on “Singing Synthesis Challenge: Fill-In the Gap”21 was organised in the context of the Interspeech2016 Conference in San Fransisco, USA on 10/9/2016.

In addition, two other events addressing the scientific community were organised:

• A Think-Tank on "Gestures and Artificial Intelligence in Art and Industry: What are the synergies after 10 years?" (GAIIA)22 on 16/1/2017 in Paris, France. The goal was to bring together professionals with research interests that involve gestures (scientists, artists, ergonomists, engineers, etc.) in order to discuss the potential synergies between art and industry.

• An event aiming to present the project to the UCL faculty and students in the fields of cultural heritage studies and digital humanities in London, UK on 1/3/2017.

Figure 41: Scientific workshops organised by i-Treasures: a) Workshop on “ICT for the Preservation and Transmission of Intangible Cultural Heritage” in Euromed2014 (Lemessos, Cyprus), b) 3rd International Symposium on Movement and Computing (MOCO’16) (Thessaloniki, Greece).

4.2.10 Organisation of events for the general public

The Consortium has organised several events targeting the general public (Figure 42). The project was presented to the audience by means of general introductory presentations and live demonstrations of the developed tools and technologies.

• TT established a Customer Experience Center in Istanbul, Turkey during 2016. In this Center, i-Treasures had its own stand, where demos and outcomes of the i-Treasures project were presented. Overall, the project results were disseminated in more than 350 visitors.

• CERTH has been organizing the European Researcher’s Night event in Thessaloniki, a popular science event organised yearly all over Europe, for more than one decade. During the last four years, i-Treasures was presented to the audience (children and adults) through interactive demonstrations of the game-like applications.

20 http://www.visapp.visigrapp.org/RGB-SpectralImaging.aspx?y=2016 21 https://chanter.limsi.fr/doku.php?id=sidebar#special_session_interspeech_2016_singing_synthesis_challenge_fill-in_the_gap 22 http://www.mines-paristech.fr/Agenda/Geste-et-intelligence-artificielle-dans-l-Art-et-l-Industrie/3773



• CERTH has been demonstrating the i-Treasures technologies (games, software for dance capture and analysis, etc.) in its annual Open Day Event for industrial partners and the general public.

• A special event for the i-Treasures project focusing on Canto a Tenore was organised for the students of a secondary school, local stakeholders and journalists, in Nuoro, Sardinia on 5/12/2014.

• A series of Masterclasses on culture and technology23 was organised in the University of Mons on 12-15/3/2014.

• Demonstration of IMI to groups of students in the Vocational Training Institute of Thermi and the Municipal Conservatory of Kalamaria in Thessaloniki, Greece on 1-2/12/2014.

Figure 42: Organisation of events for the general public: a) 2014 European

Researcher's Night in Thessaloniki, and b) CERTH’s 2015 Open day for the general public and industrial partners.

4.2.11 Participation in events

The project and its achievements were presented by the Consortium in the following events (Figure 43):

• Conferences and research events

o UNESCO Conference towards Knowledge Societies for Peace and Sustainable Development24 (Paris, France, 25-27/2/2013).

o First eCult Dialogue Day25 (Dubrovnik, Croatia, 19/9/2013).

o Digital Heritage 201326 (Marseille, France, 28/10-2/11/2013).

o ICT 2013 Event27 (Vilnius, Lithuania, 6-9/11/2013).

o 10th International Summer Workshop on Multimodal Interfaces (eNTERFACE’14)28 in (Bilbao, Spain, 9/6-4/7/2014)

o Digital Heritage 201529 (Granada, Spain, 28/9-2/10/2015).

23 http://checkthis.com/cute2014 24 http://www.unesco.org/fileadmin/MULTIMEDIA/HQ/CI/CI/pdf/wsis/WSIS_10_Event/wsis10_outcomes_en.pdf 25 http://www.ecultobservatory.eu/content/ecult-dialogue-day 26 http://www.digitalheritage2013.org/ 27 https://ec.europa.eu/digital-single-market/en/node/39748 28 http://aholab.ehu.es/eNTERFACE14



o ICT 2015 Event30 (Lisbon, Portugal, 20-22/10/2015).

o 11th International Summer Workshop on Multimodal Interfaces (eNTERFACE’15)31 (Mons, Belgium, 10/8-4/9/2015).

o Symposium on “2D and 3D Documentation and Visualisation of Performing Arts, Folklore and Rituals through the example of DANCE” organised by Dachverband Tanz Deutschland (Berlin, 16-18/11/2015).

o “Successful R&I in Europe 2015” Networking Event (Dusseldorf, Germany, 5-6/11/2015)

o CUTE 2015 masterclass (Mons, Belgium, 28-29/8/2015).

o May Residency 2015 Event32 (Vancouver, Canada, 25-30/5/2015).

o “Intangible cultural heritage and digital tools: Passing on, participative management, issues at stake”33 seminar organised by the French Center for Intangible Cultural Heritage (Vitrè, France, 8/9/2015).

o 16th International Conference on New Interfaces for Musical Expression34 (Brisbane, Australia, July 2016).

o “The CreActive Network - uno spazio per condividere e creare nuova conoscenza” Conference (Florence, Italy, 11/2017).

o 1st Conference on Computing for Social Sciences and Humanities. Talk on “New ways of safeguarding ICH through technology” given by Dr Grammalidis (Thessaloniki, 20/2/2017).

o “The Role of Museums in Promoting and Safeguarding Intangible Heritage” Workshop, co-organised by the Cyprus UNESCO Committee (Nicosia, Cyprus, 27/4/2017)

• Scientific/academic talks/lectures

o Talk on “Gesture recognition and expert gesture analysis” by Dr Joëlle Tilmanne in the 2014 Forum n Tactile and Gestural Interaction (Tourcoing, Belgium, 13/5/2014).

o Lecture on “Novel methodologies for capturing rare songs” given by Dr Lise Buchman during the World Voice Day35 (Paris, France, 15/4/2015).

o Lecture on “New technologies for capturing rare singing” given by Dr Lise Buchman in the 19th International Voice Workshop (Paris, France, 27/5/2015).

o Lecture on “Safeguarding and transmission of Corsican Cantu in Paghjella in the European FP7 i-Treasures project: a multi-sensory data collection experiment involving new capturing technologies” given by Dr Catherine Herrgott and Dr Martine Adda-Decker in the annual meeting of the Athens Institute for Education and Research (Athens, Greece, 25/6/2017)

29 http://www.digitalheritage2015.org/ 30 https://ec.europa.eu/digital-single-market/en/ict2015 31 http://www.enterface.net/enterface15/ 32 http://movingstories.ca/events/may-residency-2015/ 33 http://www.cfpci.fr/medias/PDF/2015/English_program_01_09.pdf 34 http://nime2016.org/ 35 https://www.mgen.fr/fileadmin/documents/5_Le_groupe_MGEN/Actualites/Invitation-journee-voix-2015.pdf



o Presentation on “Laryngeal gestures in Human Beatbox” at the University of Grenoble given by Dr Claire Pillot-Loiseau and Dr Lise Crevier-Buchman (France, 8/4/2016).

o Lecture on “EEG-based emotion recognition” at the Electrical and Computer Engineering Department of AUTH (Thessaloniki, Greece, 7/11/2016).

o Presentation on EEG-based emotion recognition during the 2017 Information Day of the Electrical and Computer Engineering Department of the Aristotle University of Thessaloniki (15/1/2017).

• Events for industry and the general public o MEDinART-FABRICA VITAE event (Athens, Greece, 17/12/2014). Talk

on “Conceptual blending in biomusic composition space” by Prof. L. Hadjileontiadis.

o Numediart Open Day 2014 (Mons, Belgium, 23/5/2014).

o Visit at the traditional pottery workshop of the underground Güray Ceramic Museum in Avanos, Cappadocia during the 6th i-Treasures meeting (Avanos, Turkey, 9/9/2014).

o “Quinzaine du numérique”36 event (Puppetry Museum,Tournai, Belgium, 17/10/2014)

o 1st Thessaloniki Science Festival37 (Thessaloniki, Greece, 14-17/5/2015).

o Music Forum "Lessons in a different way", State Conservatory of Thessaloniki, (28/2-8/3/2015).

o 81st International Fair of Thessaloniki (10-18/9/2016).

o Numediart open day “ICT Meets TWIST & NUMEDIART” (Mons, Belgium, 23/11/16).

36 http://quinzainenumeriquemons.be 37 https://www.thessalonikisciencefestival.gr/



Figure 43: Dissemination of i-Treasures results in research events: a) ICT 2103 (Vilnius, Lithuania), b) eNTERFACE 2014 Workshop (Bilbao, Spain), c) 1st Conference on Computing for Social Sciences and Humanities (Thessaloniki, Greece), d) ICT 2015 (Lisbon, Portugal).

4.2.12 Organisation of the i-Treasures Workshop

i-Treasures co-organised a joint Workshop on “The e-documentation of European Intangible Heritage” in cooperation with the DARIAH-CY, Terpsichore, 4D-CH-WORLD, ITN-DCH and Europeana-Space European research projects on November 4, 2016. The workshop was held in Nicosia, Cyprus within the framework of the Euromed2016 Conference on Digital Heritage38 and comprised of four parts/sessions. It was attended by approximately 50 people with research interests focused on ICT and cultural heritage (Figure 44).

The first session included short presentations on initiatives and European research projects focusing on digital intangible cultural heritage, which were given by members of our Advisory Board or other researchers. The second session included an extensive presentation of the i-Treasures results by the Consortium, covering the different research the project has focused on (educational tools, technologies for body/motion analysis in ICH expressions, serious games for ICH learning, semantic analysis, the integrated platform) and also addressing the UNESCO approach for ICH in comparison to the i-Treasures objectives. During the third session, live demonstrations of some of the developed technologies (generic game-like application for human motion capture and dance learning, TTS module for traditional singing, Web-platform) were organised. Finally, the fourth session included the presentation of several papers by other researchers, also working on the cultural heritage area. An extensive and fruitful discussion on the i-Treasures results and impact as well as on the future of ICT technologies for ICH transmission and safeguarding followed.

38 http://www.euromed2016.eu/



Figure 44: The i-Treasures Workshop on “The e-documentation of European Intangible Heritage” (Nicosia, Cyprus, 4 November 2016).

4.2.13 User group

An active group of 400 users has been established by i-Treasures partners, which includes independent experts (professional singers, dancers, potters and music composers), independent apprentices (students), pedagogical institutions (conservatoires, schools, university departments etc.), cultural heritage experts (ethnologist, UNESCO representatives) and cultural associations, interested in the four uses cases or ICH in general (Figure 45).

Figure 45: i-Treasures User Group: Number of experts/users per ICH use case.

4.2.14 Clustering activities

Clustering with other European research projects performing research on the Cultural Heritage and ICT areas has been pursued throughout the duration of the project. In this context, common areas of interest have been identified and potential fields of co-



operation and synergies with respect to these areas have been elaborated. i-Treasures partners have participated in (or organised) clustering meetings with the following EU research projects/actions, cultural heritage organisations and companies:

• ITN-DCH (Initial Training Network for Digital Cultural Heritage): http://www.itn-dch.eu

• Terpsichore (Transforming Intangible Folkloric Performing Arts into Tangible Choreographic Digital Objects): http://terpsichore-project.eu

• 4D-CH-WORLD (Four-Dimensional Cultural Heritage World): http://www.4d-ch-world.eu

• DigiArt (The Internet of Historical Things and Building New 3D cultural worlds): http://digiart-project.eu

• RePlay (Digitally capturing unique skills involved in European Traditional Sports and Games): http://fp7-replay.eu/

• ViMM (Virtual Multimodal Museum): http://www.vi-mm.eu

• V-MusT.net (Virtual Museum Transnational Network): http://www.v-must.net/

• MAXICULTURE (Maximise the impact of your digital cultural heritage project): http://www.maxiculture.eu/

• eCultValue (Valorisation of EU project results in the area of access to cultural content): https://ecultvalue.wordpress.com/

• GaLA (Games and Learning Alliance): http://www.galanoe.eu/

• Praise (Practice and peRformance Analysis Inspiring Social Education): http://www.iiia.csic.es/praise/

• Envisage (Enhance virtual learning spaces using applied gaming in education): http://www.envisage-h2020.eu

• SKAT-VG (Sketching Audio Technologies using Vocalizations and Gestures) http://skatvg.iuav.it/

• Pericles (Promoting and Enhancing Reuse of Information throughout the Content Lifecycle taking account of Evolving Semantics): http://pericles-project.eu

• 3D-ConTourNet (3D Content Creation, Coding and Transmission over Future Media Networks): http://www.3d-contournet.eu/

• HDR (Digital Capture, Storage, Transmission and Display of Real - World Lighting): http://www.ic1005-hdri.com/

• Moving Stories: http://movingstories.ca/

• Europa Nostra: http://www.europanostra.org/

• FARO (Flemish Interface Cantre for Cultural Heritage): http://www.faronet.be

• Dachverband Tanz Germany (National Association of Dance of Germany)

• Working Group V/2 of ISPRS (International Society for Photogrammetry and Remote Sensing) on “Cultural heritage data acquisition and processing” http://www2.isprs.org/commissions/comm5/wg2.html

• Wekinator: http://www.wekinator.org/

• MuseScore: https://musescore.org/



Finally, members of consortium visited the Ecumenical Patriarchate of Constantinople in Istanbul, Turkey and met His All Holiness, Bartholomew, Archbishop of Constantinople, New Rome and Ecumenical Patriarch on 22/9/2012. They informed the Ecumenical Orthodox Patriarch about the main objectives of the i-Treasures project and its potential impact on the preservation of intangible cultural heritage. The discussion focused on the proposed technology for the capture and analysis of rare traditional songs and particularly of Byzantine hymns.

4.2.15 Local training workshops

To increase the impact of the i-Treasures technologies on the cultural heritage community, three local training workshops aimed at safeguarding local expressions of intangible heritage and contributing to local development strategies were organised in Thessaloniki, Mons and Corsica in 2016 (Figure 46). These were addressed to education and culture professionals as well as community members with an interest in heritage preservation, new technologies and local development. The main objective was to equip participants with knowledge and skills that will allow them to design and implement activities for safeguarding expressions of intangible heritage in their communities. About 100 people participated in these workshops and some of them planned and delivered their own small-scale safeguarding projects for local ICH expressions, relating to local museums, cultural associations and community initiatives and using technologies developed within i-Treasures, under the guidance of Consortium members.

Figure 46: Local training workshops for ICH safeguarding organised at a) the Folk Art and Ethnological Museum of Macedonia and Thrace in Thessaloniki, b) the Museum of

Corte in Corsica, and c) the University of Mons.

Moreover, a course in a multi-disciplinary Master programme on “Educational Sciences - Learning Technologies”39, offered by the Aristotle University of Thessaloniki, was structured around i-Treasures tools and methodologies and included several lectures by i-Treasures researchers. The students were invited to submit a small safeguarding project as part of their assessment, which consisted of the design and implementation of an ICH Safeguarding Plan and required active collaboration with local communities and combined use of i-Treasures technologies, like the pedagogical planner and the LMS, and relevant ethnographic methods. 11 projects were submitted by the student, focusing in traditional dances from Northern Greece and Cappadocia (Zonaradikos, Moulaevo, Koniali), the intangible heritage of entire local communities (Sirrako in Epirus and Asvestades in Evros), traditional

39 MSc programme on“Educational Sciences - Learning Technologies” in the Aristotle University of Thessaloniki: http://learntech.web.auth.gr/learntech/



craftsmanship (decoupage, woodcrafting, tsarouhia), performing arts (puppetry, Greek shadows theatre - Karagiozis) and chess opening moves( Figure 47).

Figure 47: LMS courses / safeguarding projects developed by students of the cross-disciplinary MA programme of AUTH “Educational Sciences - Learning Technologies”: a) Koniali, a traditional dance from Cappadocia also known as the “dance of spoons”, b) the making of tsarouhi, a type of Greek traditional rural shoe.

The training workshops have been a positive experience not only for the participants but also for the Consortium and offered the opportunity to establish an i-Treasures community of practitioners, students and professionals. The aims were met with success not only in terms of raising awareness about the i-Treasures Web-platform and technologies, but more importantly by providing participants with research skills and hands-on experience of safeguarding intangible heritage with the input of new technologies.

4.2.16 Publications

Results from the research carried out within the i-Treasures project have been published in international conferences and scientific journals. The partners published 19 papers in peer-reviewed journals, 1 book, 3 book chapters and 73 papers in conferences proceedings (96 in total). Four more journal papers have been submitted for publication. A full list of i-Treasures publications is presented below.

Project publications are available to the general public through the Publications page of the project Website40. In this page, links to camera-ready versions of these papers (pdf files) are available, which redirect the user to institutional repositories of partners or open access repositories. Permission to make the publications open access was requested by journal and conference proceedings publishers where required.

Journal papers 1. K. Dimitropoulos, A. Kitsikidis, and N. Grammalidis, "Information technologies in

traditional dance analysis", Journal of Science of Dance, April-May 2014.

2. F. M. Dagnino, L. Hadjileontiadis, M. Ott, and F. Pozzi, “An integrated platform supporting Intangible Cultural Heritage learning and transmission: definition of

40 i-Treasures publications: http://www.i-treasures.eu/content/publications-0



requirements and evaluation criteria”, Journal of Computing and Information Technology (CIT), Vol. 22, No 4, pp. 277–292, 2014.

3. F. Pozzi, M. Alivizatou, F. M. Dagnino, and M. Ott, “Going Beyond Preservation: How to Support Technology-Enhanced Learning in ICH Education”, International Journal of Heritage in the Digital Era, Vol. 4, No 1, pp. 21-40, March 2015.

4. M. Ott, F. M. Dagnino, and F. Pozzi, “Intangible Cultural Heritage: Towards Collaborative Planning of Educational Interventions”, Computers in Human Behaviour, Vol. 51, Part B, pp. 1314–1319, October 2015.

5. Sofia B. Dias, Sofia J. Hadjileontiadou, Leontios J. Hadjileontiadis, and Jose A. Diniz, "Fuzzy Cognitive Mapping of LMS Users’ Quality of Interaction within Higher Education Blended-Learning Environment", Expert Systems with Applications, Vol. 42, No. 21, pp. 7399–7423, November 2015.

6. S. Hadjidimitriou, V. Charisis and L. Hadjileontiadis, “Towards a practical subject iIndependent affective state recognition based on time-domain EEG feature extraction”, International Journal of Heritage in the Digital Era, Vol. 4, No. 2, pp. 165-177, June 2015.

7. F. M. Dagnino, M. Ott, and F. Pozzi, “Addressing key challenges in Intangible Cultural Heritage education”, International Journal of Heritage in the Digital Era, Vol. 4, No. 2, pp. 193 - 207 , June 2015.

8. L. Crevier-Buchman, A. Amelot, S. K. Al Kork, M. Adda-Decker, N. Audibert, P. Chawah, B. Denby, T. Fux, A. Jaumard-Hakoun, P. Roussel, M. Stone, J. Vaissiere, K. Xu, and C. Pillot-Loiseau, “Acoustic data analysis from multi-sensor capture in rare singing: Cantu in Paghjella case study”, International Journal of Heritage in the Digital Era, Vol. 4, No. 1, pp. 121-132, June 2015.

9. A. Kitsikidis, N. V. Boulgouris, K. Dimitropoulos, and N. Grammalidis, "Unsupervised dance motion patterns classification from fused skeletal data using exemplar-based HMMs", International Journal of Heritage in the Digital Era, Vol. 4, No. 2, pp. 209-220, June 2015.

10. K. Xu, Y. Yang, A. Jaumard-Hakoun, C. Leboullenger, G. Dreyfus, P. Roussel, M. Stone and B. Denby, “Robust contour tracking in ultrasound tongue image sequences”, Clinical Linguistics & Phonetics, Vol. 30, No. 3-5, pp. 313-327, March-May 2016.

11. K. Xu, T. Gábor Csapó, P. Roussel, and B. Denby, “A comparative study on the contour tracking algorithms in ultrasound tongue images with automatic re-initialization”, Journal of the Acoustical Society of America, Vol. 139, No. 5, pp. 154-160, May 2016.

12. S. Laraba and J. Tilmanne. “Dance performance evaluation using Hidden Markov Models”, Computer Animation and Virtual Worlds Journal (CASA), Vol. 27, No. 3-4, pp. 321–329, May-August 2016

13. E. Chatzilari, S. Nikolopoulos, and I. Kompatsiaris, “SALIC: Social Active Learning for Image Classification”, IEEE Transactions on Multimedia, Vol. 18, No. 8, pp. 1488-1503, August 2016.

14. F. Pozzi, A. Ceregini, F. M. Dagnino, M. Ott and M. Tavella “Closing the "learning design lifecycle" with the Pedagogical Planner”, Special issue of European Journal of Open, Distance and E-Learning (EURODL), pp. 103-116, December 2016.

15. G. Cozzani, F. Pozzi, F. M. Dagnino, A. V. Katos, and E. F. Katsouli, “Innovative technologies for intangible cultural heritage education and preservation: the case



of i-Treasures”, Personal and Ubiquitous Computing, Vol. 21 No 2, pp. 253-265, April 2017.

16. Kele Xu, P. Roussel, T. G. Csapó, and B. Denby, “Convolutional neural network-based automatic classification of midsagittal tongue gestural targets using B-mode ultrasound images”, Journal of the Acoustical Society of America, Vol. 141, No. 6, June 2017.

17. K. Dimitropoulos, A. Kitsikidis P. Barmpoutis, and N. Grammalidis, “Classification of multidimensional time-evolving data using histograms of Grassmannian points”, to appear in IEEE Transactions on Circuits and Systems for Video Technology, 2017.

18. I. Poulios, “Managing ‘difficult’ intangible heritage through the application of the living heritage approach: reflections on the documentary film “Silent Witness” on the prison of the city of Trikala in Greece”, to appear in MuseumEdu, Special Issue on Museum Education and ‘Difficult Heritage’, 2017.

19. C. Volioti, S. Manitsaris, E. Hemery, S. Hadjidimitriou, V. Charisis, L. Hadjileontiadis, E. Katsouli, F. Moutarde and A. Manitsaris, “A natural user interface for gestural expression and emotional elicitation to access the musical Intangible Cultural Heritage”, to appear in Journal on Computing and Cultural Heritage (JOCCH), ACM, 2017.

20. G. Chantas, S. Karavarsamis, S. Nikolopoulos, and I. Kompatsiaris, “A probabilistic, ontological framework for safeguarding the Intangible Cultural Heritage”, Journal on Computing and Cultural Heritage, 2017, submitted for publication.

21. P. Kritopoulou, M. Lazaridou, S. Manitsaris, A. Glushkova, C.Volioti, E. Katsouli, and A. Manitsaris, “A sensorimotor e-learning platform for the transmission of technical and artistic motor skills using motion capture”, Journal of Computer Assisted Learning, 2017, submitted for publication.

22. K. Dimitropoulοs, S. Manitsaris, F. Tsalakanidou, B. Denby, L. Crevier-Buchman, S. Dupont, S. Nikolopoulos, Y. Kompatsiaris, V. Charisis, L. Hadjileontiadis, F. Pozzi, M. Cotescu, S. Ciftsi, A. Katos, A. Manitsaris and N. Grammalidis, “A multimodal approach for the safeguarding and transmission of Intangible Cultural Heritage: The case of i-Treasures”, IEEE Intelligent Systems, 2017, submitted for publication.

23. V. Charisis, S. Hadjidimitriou and L.Hadjileontiadis, "FISEVAL-A novel fuzzy logic-based project evaluation approach: The paradigm of the i-Treasures project", International Journal of Project Management, 2017, submitted for publication.

Books 1. S. J. Hadjileontiadou, S. B. Dias, J. A. Diniz, and L. J. Hadjileontiadis, Fuzzy

Logic-Based Modeling in Collaborative and Blended Learning, IGI Global, July 2015.

Book chapters 1. S. B. Dias, S. J. Hadjileontiadou, J. A. Diniz, and L. J. Hadjileontiadis, "Towards a

Personalized and Intelligent Web 3.0 Hybrid Learning Environment via the Quality of Collaboration and Interaction Modeling: A Fuzzy Logic-Based Approach", in Emerging Web 3.0/Semantic Web Applications in Higher Education: Growing



Personalization and Wider Interconnections in Learning (Research in Management Education and Development). C. Wankel & A. Stachowicz-Stanusch (Eds.), Information Age Publishing (IAP), New York, September 2015.

2. F.M. Dagnino and F. Pozzi, “The i-Treasures project: capturing the intangible through Information and Communication Technologies”, in Marta Severo and Séverine Cachat (Eds) Patrimoine culturel immatériel et numérique: transmission, participation, enjeux, L’Harmattan, September 2016.

3. M. Alivizatou-Barakou, A. Kitsikidis, F. Tsalakanidou, K. Dimitropoulos, G. Chantas, S. Nikolopoulos, S. Al Kork, B. Denby, L. Buchman, M. Adda-Decker, C. Pillot-Loiseau, J. Tillmane, S. Dupont, B. Picart, F. Pozzi, M. Ott, E. Yilmaz, V. Charisis, S. Hadjidimitriou, L. Hadjileontiadis, M.Cotescu, C.Volioti, A. Manitsaris, S. Manitsaris, and N. Grammalidis, “Intangible Cultural Heritage and new technologies: Challenges and opportunities for cultural preservation and development”, in M. Ioannides, N. Magnenat-Thalmann, and G. Papagiannakis (Eds), Mixed Reality and Gamification for Cultural Heritage, Springer International Publishing, February 2017.

Conference papers 1. D. Mandiliotis, K. Toumpas, K. Kyprioti, K. Kaza, J. Barroso, and L. J.

Hadjileontiadis, “Symbiosis: An Innovative Human-Computer Interaction Environment for Alzheimer's Support”, in Proc. 15th International Conference on Human-Computer Interaction, Las Vegas, Nevada, USA, 21-26 July, 2013.

2. N. d’Alessandro, J. Tilmanne, M. Astrinaki, T. Hueber, R. Dall, T. Ravet, A. Moinet, H. Cakmak, O. Babacan, A. Barbulescu, V. Parfait, V. Huguenin, E. Sumeyye Kalayci, and Qiong Hu, “Reactive Statistical Mapping: Towards the Sketching of Performative Control with Data”, in Proc. 9th International Summer Workshop on Multimodal Interfaces (eNTERFACE'13), Lisbon, Portugal, July 15th- August 9th 2013.

3. A. Dapogny, R. de Charette, S. Manitsaris, F. Moutarde, and A. Glushkova, "Towards a Hand Skeletal Model for Depth Images Applied to Capture Music-like Finger Gestures", in Proc. 10th International Symposium on Computer Music Multidisciplinary Research (CMMR 2013), Marseille, France, 15-18 October 2013.

4. A. Antonaci, F. Dagnino, M. Ott, F. Pozzi, P. Bravi, and M. Lutzu, “Digital Technology and Transmission of Intangible Cultural Heritage: the Case of Cantu a Tenore”, in Digital Heritage 2013, Marseille, France, October 28th-November 1st 2013.

5. A. Jaumard-Hakoun, S. K. Al Kork, M. Adda-Decker, A. Amelot, L. Buchman, T. Fux, C. Pillot, P. Roussel, M. Stone, G. Dreyfus, and B. Denby, “Capturing, Analyzing, and Transmitting Intangible Cultural Heritage with the i-Treasures Project”, in Proc. Ultrafest VI, Edinburgh, UK, 6-8 November 2013.

6. A. Antonaci, P. Bravi, F. Dagnino, M. Lutzu, M. Ott, S. Pilosu, and F. Pozzi, “Exploring the “Intangible” Through ICT”, in Proc. 6th International Conference of Education, Research and Innovation (ICERI2013), Seville, Spain, 18-20 November 2013.

7. A. Kitsikidis, K. Dimitropoulos, S. Douka, and N. Grammalidis, “Dance Analysis using Multiple Kinect Sensors”, in Proc. 9th International Conference on Computer Vision Theory and Applications (VISAPP2014), Lisbon, Portugal, 5-8 January 2014.



8. K. Dimitropoulos, S. Manitsaris, F. Tsalakanidou, S. Nikolopoulos, B. Denby, S. Al Kork, L. Crevier-Buchman, C. Pillot-Loiseau, S. Dupont, J. Tilmanne, M. Ott, M. Alivizatou, E. Yilmaz, L. Hadjileontiadis, V. Charisis, O. Deroo, A. Manitsaris, I. Kompatsiaris, and N. Grammalidis, “Capturing the Intangible: An Introduction to the i-Treasures Project”, in Proc. 9th International Conference on Computer Vision Theory and Applications (VISAPP2014), Lisbon, Portugal, 5-8 January 2014.

9. F. Pozzi, A. Antonaci, F. M. Dagnino, M. Ott, and M. Tavella, “A Participatory Approach to Define User Requirements of a Platform for Intangible Cultural Heritage Education”, in Proc. 9th International Conference on Computer Vision Theory and Applications (VISAPP2014), Lisbon, Portugal, 5-8 January 2014.

10. G. Chantas, S. Nikolopoulos, and I. Kompatsiaris, “Multi-entity Bayesian Networks for Treasuring the Intangible Cultural Heritage”, in Proc. 9th International Conference on Computer Vision Theory and Applications (VISAPP2014), Lisbon, Portugal, 5-8 January 2014.

11. S. B. Dias, L. J. Hadjileontiadis, and J. A. Diniz, “On Enhancing Blended-Learning Scenarios through Fuzzy Logic-based Modeling of Users’ LMS Quality of Interaction - The Rare & Contemporary Dance Paradigms”, in Proc. 9th International Conference on Computer Vision Theory and Applications (VISAPP2014), Lisbon, Portugal, 5-8 January, 2014.

12. J. Tilmanne, N. d’Alessandro, M. Astrinaki and T. Ravet, “Exploration of a Stylistic Motion Space Through Realtime Synthesis”, in Proc. 9th International Conference on Computer Vision Theory and Applications (VISAPP2014), Lisbon, Portugal, 5-8 January 2014.

13. F. Pozzi, F. M. Dagnino, M. Ott and A. Antonaci “i-Treasures and Intangible Cultural Heritage education”, in Proc. Museums and the Web Florence International Conference (MWF2014), Pre-conference workshop, Florence, Italy, 18-21 February 2014.

14. J. Tilmanne, N. D'Alessandro, T. Ravet, M. Astrinaki, and A. Moinet, "Full-body gait reconstruction using covariance-based mapping within a realtime HMM-based framework", in Proc. 50th Annual Convention of the AISB (AISB-50), pp. 313-316, London, UK, 1 - 4 April 2014.

15. S. K. Al Kork, A. Jaumard-Hakoun, M. Adda-Decker, A. Amelot, L. Buchman, T. Fux, C. Pillot, P. Roussel, M. Stone, G. Dreyfus, and B. Denby, "A multi-sensor hyper-helmet to capture rare singing, an Intangible Cultural Heritage study", in Proc. 10th International Seminar on Speech Production (ISSP2014), Cologne, Germany, 5 - 8 May 2014.

16. T. Fux, A. Amelot, L. Crevier-Buchman, C. Pillot-Loiseau, and M. Adda-Decker, "On the use of accelerometer sensors to study nasality in speech and singing voice", in Proc. 10th International Seminar on Speech Production (ISSP2014), Cologne, Germany, 5 - 8 May 2014.

17. L. Crevier-Buchman, T. Fux, C. Pillot-Loiseau, A. Amelot, and M. Adda-Decker, "Measuring vocal tract agility from videofiberscopic data: The Human BeatBox example", in Proc. The Voice Foundation 2014 - 43rd Annual Symposium: Care of the Professional Voice, Philadelphia, USA, 28 May - 1 June 2014.

18. T. Ravet, J. Tilmanne, and N. D'Alessandro, "Hidden Markov Model based real-time motion recognition and following", in Proc. International Workshop on Movement and Computing (MOCO'14), Paris, France, 16-17 June 2014.

19. C. Volioti, S. Manitsaris, and A. Manitsaris, "Offline statistical analysis of gestural



skills in pottery interaction", in Proc. International Workshop on Movement and Computing (MOCO'14), Paris, France, 16-17 June 2014.

20. S. B. Dias, J. A. Diniz, and L. J. Hadjileontiadis, "Exploring B-learning scenarios using fuzzy logic-based modeling of users’ LMS quality of interaction in ergonomics and psychomotor rehabilitation academic courses", in Proc. HCI International 2014, Heraklion, Greece, 22-27 June 2014.

21. A. Kitsikidis, K. Dimitropoulos, E. Yilmaz, S. Douka and N. Grammalidis, "Multi-sensor technology and fuzzy logic for dancer’s motion analysis and performance evaluation within a 3D virtual environment", in Proc. HCI International 2014, Heraklion, Greece, 22-27 June 2014.

22. M. Ott, F. M. Dagnino, F. Pozzi, and M. Tavella, "Widening access to Intangible Cultural Heritage: Towards the development of an innovative platform", in Proc. HCI International 2014, Heraklion, Greece, 22-27 June 2014.

23. S. B. Dias, J. A. Diniz, and L. J. Hadjileontiadis, "Dynamic fuzzy logic-based quality of interaction within blended learning: The rare and contemporary dance cases", in Proc. 8th International Conference on e-Learning, Lisbon, Portugal, 15-18 July 2014.

24. G. Chantas, A. Kitsikidis, S. Nikolopoulos, K. Dimitropoulos, S. Douka, I. Kompatsiaris, and N. Grammalidis, "Multi-entity Bayesian networks for knowledge-driven analysis of ICH content", in Proc. 1st International Workshop on Computer vision + ONTology Applied Cross-disciplinary Technologies (CONTACT 2014) - 13th European Conference on Computer Vision (ECCV 2014), Zurich, Switzerland, 7 September 2014. This paper received the Best student paper award.

25. Kele Xu, Yin Yang, A. Jaumard-Hakoun, M. Adda-Decker, A. Amelot, S. K. Al Kork, L. Crevier-Buchman, P. Chawah, G. Dreyfus, T. Fux, C. Pillot-Loiseau, P. Roussel, M. Stone, and B. Denby, "3D tongue motion visualization based on ultrasound image sequences", in Proc. 15th Annual Conference of the International Speech Communication Association (Interspeech 2014), Singapore, 14-18 September 2014.

26. P. Chawah, S. K. Al Kork, T. Fux, M. Adda-Decker, A. Amelot, N. Audibert, B. Denby, G. Dreyfus, A. Jaumard-Hakoun, C. Pillot-Loiseau, P. Roussel, M. Stone, K. Xu, and L. Buchman, "An educational platform to capture, visualize and analyze rare singing", in Proc. 15th Annual Conference of the International Speech Communication Association (Interspeech 2014), Singapore, 14-18 September 2014.

27. L. J. Hadjileontiadis, “Conceptual blending in biomusic composition space: the “Brainswarm” paradigm”, in Proc. Joint 11th Sound and Music Computing Conference and 40th International Computer Music Conference, Athens, Greece, 14 -20 September 2014.

28. T. Fux, A. Amelot, L. Crevier-Buchman, C. Pillot-Loiseau and M. Adda-Decker, "MATTONG: Une interface graphique sous MatLab pour le suivi du contour de la langue à partir d’images ultrasons", in JEP 2014 (Journée d'Etude sur la Parole), Le Mans, France, 20-21 September 2014.

29. E. Chatzilari, S. Nikolopoulos, Y. Kompatsiaris, and J. Kittler, "How many more images do we need? Performance prediction of bootstrapping for image classification", in Proc. 21st IEEE International Conference on Image Processing (ICIP2014), Paris, France, 27-30 October 2014.



30. F. M. Dagnino, M. Ott, F. Pozzi, E. Yilmaz, F. Tsalakanidou, K. Dimitropoulos, and N. Grammalidis, "Serious games to support learning of rare ‘intangible’ cultural expressions", in Proc. 9th International Technology, Education and Development Conference (INTED2015), pp. 7184-7194, Madrid, Spain, 2-4 March 2015.

31. E. Yilmaz, D. Uğurca, C. Şahin, F. M. Dagnino, M. Ott, F. Pozzi, K. Dimitropoulos, F. Tsalakanidou, A. Kitsikidis, S. K. Al Kork, K. Xu, B. Denby, P. Roussel, P. Chawah, L. Buchman, M. Adda-Decker, S. Dupont, B. Picart, J. Tilmanne, M. Alivizatou, L. Hadjileontiadis, V. Charisis, A. Glushkova, C. Volioti, A. Manitsaris, E. Hemery, F. Moutarde, and N. Grammalidis, “Novel 3D Game-like Applications Driven by Body Interactions for Learning Specific Forms of Intangible Cultural Heritage”, in Proc. 10th International Conference on Computer Vision Theory and Applications (VISAPP2015), Berlin, Germany, 11-14 March 2015.

32. S. K. Al Kork, D. Uğurca, C. Şahin, P. Chawah, L. Buchman, M. Adda-Decker , K. Xu, B. Denby, P. Roussel, B. Picart, S. Dupont, F. Tsalakanidou, A. Kitsikidis, F. Dagnino, M. Ott, F. Pozzi , M. Stone, and E. Yilmaz, “A Novel Human Interaction Game-Like application to Learn, Perform and Evaluate Modern Contemporary Singing: “ Human Beat Box””, in Proc. 10th International Conference on Computer Vision Theory and Applications (VISAPP2015), Berlin, Germany, 11-14 March 2015.

33. B. Picart, S. Brognaux, and S. Dupont, "Analysis and automatic recognition of human beatbox sounds: A comparative study", in Proc. 40th IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2015), Brisbane, Australia, 19-24 April 2015.

34. F. M. Dagnino, M. Ott, F. Pozzi, and E. Yilmaz, "Serious Games design: reflections from an experience in the field of Intangible Cultural Heritage education", in Proc. 10th E-learning and Software for Education Conference (eLSE), Bucharest, Romania, 23-24 April 2015.

35. A. Glushkova, E. Katsouli, G. Kourvoulis, A. Manitsaris, and C. Volioti, "A Hybrid Content-Learning Management System for Education and Access to Intangible Cultural Heritage", in Proc. of the 17th International Conference on Computer Supported Education (CSEDU 2015), Lisbon, Portugal, 23-25 May 2015.

36. S. B. Dias, S. Hadjileontiadou, J. A. Diniz, and L. J. Hadjileontiadis, “Towards a Hybrid World: The Fuzzy Quality of Collaboration/Interaction (FUZZYQOC/I) Hybrid Model in the Semantic Web 3.0”, in Proc. 7th International Conference on Computer Supported Education (CSEDU 2015), Lisbon, Portugal, 23-25 May 2015.

37. N. d’Alessandro, J. Tilmanne, A. Moreau, and A. Puleo, "AirPiano: A Multi-Touch Keyboard with Hovering Control", in Proc. 15th International Conference on New Interfaces for Musical Expression (NIME2015), Baton Rouge, Louisiana, USA, 31 May - 3 June 2015.

38. S. Manitsaris, A. Glushkova, E. Katsouli, A. Manitsaris and C. Volioti, "Modelling gestural know-how in pottery based on state-space estimation and system dynamic simulation", in Proc. 6th International Conference on Applied Human Factors and Ergonomics (AHFE 2015), Las Vegas, USA, 26-30 July 2015.

39. C.Volioti, E. Hemery, S. Manitsaris, V. Tsekouropoulou, E. Yilmaz, F. Moutarde, and A. Manitsaris, “Music gestural skills development engaging teachers, learners and expert performers”, in Proc. 6th International Conference on Applied Human Factors and Ergonomics (AHFE 2015), 26-30 July 2015, Las Vegas, USA.



40. E. Hemery, S. Manitsaris, F. Moutarde, C. Volioti, and A. Manitsaris, “Towards the design of a natural user interface for performing and learning musical gestures”, in Proc. 6th International Conference on Applied Human Factors and Ergonomics (AHFE 2015), 26-30 July 2015, Las Vegas, USA.

41. S. Manitsaris, G. Milona, and A. Glushkova, “Designing optical sensorimotor feedback for GUIs supporting the learning of gestural know – how in pottery”, in Proc. 6th International Conference on Applied Human Factors and Ergonomics (AHFE 2015), 26-30 July 2015, Las Vegas, USA.

42. J. Tilmanne, N. d'Alessandro, P. Barborka, F. Bayansar, F. Bernardo, R. Fiebrink, A. Heloir, E. Hemery, S. Laraba, A. Moinet, F. Nunnari, T. Ravet, L. Reboursière, A. Sarasua, M. Tits, N. Tits, and F. Zajéga, “Prototyping a New Audio-Visual Instrument Based on Extraction of High-Level Features on Full-Body Motion”, in Proc. 10th International Summer Workshop on Multimodal Interfaces (eNTERFACE'15), Mons, Belgium, August 2015.

43. S. B. Dias, S. Hadjileontiadou, J. A. Diniz, and L. J. Hadjileontiadis, “Enhancing Blended Environments through Fuzzy Cognitive Mapping of LMS users’ Quality of Interaction: The Rare & Contemporary Dance Paradigms”, in Proc. HCI International 2015, Los Angeles, USA, 2-7 August 2015.

44. V. Charisis, S. Hadjidimitriou, L. Hadjileontiadis, D. Ugurca, and E. Yilmaz, "EmoActivity - An EEG-based gamified emotion HCI for augmented artistic expression: The i-Treasures paradigm", in Proc. HCI International 2015, Los Angeles, USA, 2-7 August 2015.

45. A. Kitsikidis, K. Dimitropoulos, D. Uğurca, C. Baycay, E. Yilmaz, F. Tsalakanidou, S. Douka, and N. Grammalidis, "A Game-like Application for Dance Learning using a Natural Human Computer Interface", in Proc. HCI International 2015, Los Angeles, USA, 2-7 August 2015.

46. A. Jaumard-Hakoun, K. Xu, P. Roussel-Ragot, G. Dreyfus, M. Stone and B. Denby, “Tongue contour extraction from ultrasound images based on Deep Neural Network”, in Proc. 18th International Congress on Phonetic Sciences (ICPhS 2015), Glasgow, UK, 10-14 August 2015.

47. K. Xu, Y. Yang, A. Jaumard-Hakoun, C. Leboullenger, G. Dreyfus, P. Roussel, M. Stone and B. Denby, “Development of a 3D tongue motion visualization platform based on ultrasound image sequences”, in Proc. 18th International Congress on Phonetic Sciences (ICPhS 2015), Glasgow, UK, 10-14 August 2015.

48. J. Tilmanne and N. d’Alessandro, “MotionMachine: A new framework for motion capture signal feature prototyping”, in Proc. 2015 European Signal Processing Conference (EUSIPCO 2015), Nice, France, 31 August – 4 September 2015.

49. N. Grammalidis, K. Dimitropoulos, A. Kitsikidis, M. Politou, E. Yilmaz, D. Ugurca, C. Sahin, F.M. Dagnino, M. Ott, F. Pozzi, S. K. Al Kork, K. Xu, B. Denby, P. Roussel, P. Chawah, L. Buchman, M. Adda-Decker, S. Dupont, B. Picart, J. Tilmanne, M. Alivizatou, L. Hadjileontiadis, V. Charisis, S. Hadjidimitriou, A. Glushkova, C. Violioti, A. Manitsaris, E. Hemery, and F. Moutarde, “Novel 3-D game applications for Intangible Cultural Heritage”, in Proc. Pan-Hellenic Conference on Digital Cultural Heritage 2015, Volos, Greece, 24-26 September 2015.

50. F. M. Dagnino, M. Ott, F. Pozzi, E. Yilmaz, K. Dimitropoulos, N. Grammalidis, and F. Tsalakanidou, “Designing Serious Games for ICH Education”, in Proc. Digital Heritage Conference 2015 (DH’15), Granada, Spain, 28 September-2 October 2015.



51. S. Laraba, J. Tilmanne, and T. Dutoit, “Adaptation procedure for HMM-based sensor-dependent gesture recognition”, in Proc. 8th ACM SIGGRAPH Conference on Motion in Games (MIG2015), Paris, France, 16-18 November 2015.

52. F. Pozzi, A. Ceregini, F. M. Dagnino, M. Ott and M. Tavella, “Supporting the ‘whole learning design life-cycle’ through the pedagogical planner”, in Proc. 1st D4Learning International Conference Innovations with Digital Learning for Inclusion (D4L), pp. 90-97, Aalborg, Denmark, 17-20 November 2015. The paper was awarded with the Best Paper Award.

53. K. Dimitropoulos, P. Barmpoutis, A. Kitsikidis, and N. Grammalidis, “Extracting dynamics from multi-dimensional time-evolving data using a bag of higher-order Linear Dynamical Systems”, in Proc. 11th International Conference on Computer Vision Theory and Applications (VISAPP 2016), Rome, Italy, 27-29 February 2016.

54. K. Xu, Y. Yang, C. Leboullenger, P. Roussel, and B. Denby, “Contour-based 3D tongue motion visualization using the ultrasound image sequences”, in Proc. 41st IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2016), Shanghai, China, 20-25 March 2016.

55. T. Ravet, N. d’Alessandro, J. Tilmanne, and S. Laraba, “Motion Data and Machine Learning: Prototyping and Evaluation”, in Proc. ACM CHI Conference on Human Factors in Computing Systems 2016 (ACM CHI 2016), San Jose, USA, 7-12 May 2016.

56. S. Karavarsamis, G. Chantas, D. Ververdidis, S. Nikolopoulos, and I. Kompatsiaris, "Classifying Salsa dance steps from skeleton poses", in Proc. 14th International Workshop on Content-based Multimedia Indexing (CBMI 2016), Bucharest, Romania, 15-17 June 2016.

57. C. Pillot-Loiseau, P. Chawah, A. Amelot, G. Bachman, C. Herrgott, M. Adda-Decker and L. Crevier-Buchman, “Production des voyelles parlées et chantées dans le Cantu in Paghjella” (“Specificities in spoken and sung vowels in Cantu in Paghjella”), in Proc. 31st Journées d'Etude sur la Parole (JEP), Paris, France, 4-8 July 2016, pp. 545-553.

58. N. Grammalidis, K. Dimitropoulos, F. Tsalakanidou, A. Kitsikidis, P. Roussel, B. Denby, P. Chawah, L. Buchman, S. Dupont, S. Laraba, B. Picart, M. Tits, J. Tillmane, S. Hadjidimitriou, L. Hadjileontiadis, V. Charisis, C. Volioti, A. Stergiaki, A. Manitsaris, O. Bouzos, and S. Manitsaris, “The i-Treasures Intangible Cultural Heritage dataset”, in Proc. 3rd International Workshop on Movement and Computing (MOCO’16), Thessaloniki, Greece, 5-6 July 2016.

59. M. Tits, J. Tilmanne, and N. d'Alessandro, “A novel tool for motion capture database factor statistical exploration”, in Proc. 3rd International Workshop on Movement and Computing (MOCO’16), Thessaloniki, Greece, 5-6 July 2016.

60. E. Hemery, S. Manitsaris, and F. Moutarde, “A tabletop instrument for manipulation of sound morphologies with hands, fingertips and upper-body”, in Proc. 3rd International Workshop on Movement and Computing (MOCO’16), Thessaloniki, Greece, 5-6 July 2016.

61. C. Volioti, S. Hadjidimitriou, S. Manitsaris, L. Hadjileontiadis, V. Charisis, and A. Manitsaris, “On mapping emotional states and implicit gestures to sonification output from the “Intangible Musical Instrument””, in Proc. 3rd International Workshop on Movement and Computing (MOCO’16), Thessaloniki, Greece, 5-6 July 2016.



62. D. Ververidis, S. Karavarsamis, S. Nikolopoulos, and I. Kompatsiaris, “Pottery gestures style comparison by exploiting MYO sensor and forearm anatomy”, in Proc. 3rd International Workshop on Movement and Computing (MOCO’16), Thessaloniki, Greece, 5-6 July 2016.

63. O. Bouzos, Y. Jacob, S. Manitsaris and A. Glushkova, “3D-scene modelling of professional gestures when interacting with moving, deformable and revolving objects”, in Proc. 3rd International Workshop on Movement and Computing (MOCO’16), Thessaloniki, Greece, 5-6 July 2016.

64. K. Kokkinidis, A. Stergiaki and A. Tsagaris, “Error proving and sensorimotor feedback for singing voice”, in Proc. 3rd International Workshop on Movement and Computing (MOCO’16), Thessaloniki, Greece, 5-6 July 2016.

65. C. Volioti, S Manitsaris, E. Katsouli, A. Manitsaris, “x2Gesture: how machines could learn expressive gesture variations of expert musicians”, in Proc. 16th International Conference on New Interfaces for Musical Expression (NIME2016), Brisbane, Australia, 11-15 July 2016.

66. A. Jaumard-Hakoun, Kele Xu, C. Leboullenger, P. Roussel, and B. Denby “An articulatory-based singing voice synthesis using tongue and lips imaging”, in Proc. 17th Annual Conference of the International Speech Communication Association (InterSpeech2016), San Francisco, USA, 8-12 September 2016.

67. M. Cotescu, “Optimal unit stitching in a unit selection singing synthesis system”, in Proc. 17th Annual Conference of the International Speech Communication Association (InterSpeech2016), San Francisco, USA, 8-12 September 2016, pp. 1255-1259.

68. C. Hergott, “Cantu in Paghjella: Patrimoine Culturel Immatériel et nouvelles technologies dans le projet i-Treasures” (“Cantu in Paghjella: Intangible Cultural Heritage and new technologies in i-Treasures project”), in Proc. Patrimoines documentaires communs et analyses des cultures en mouvement: Nouvelle Aquitaine, Amérique du Nord francophone (Shared documentary heritage and the analysis of cultures in movement - New Aquitaine and French-speaking North America), Poitiers, France, 16-18 November 2016.

69. F. M. Dagnino, F. Pozzi, G. Cozzani and L. Bernava, “Using serious games for Intangible Cultural Heritage (ICH) education: A journey into the Canto a Tenore singing style”, in Proc. 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2017), Porto, Portugal, 27 February-1 March 2017, pp. 429-435.

70. K. Kokkinidis, A. Stergiaki, and A. Tsagaris, “Machine learning via multimodal signal processing”, in Proc. 6th International Conference on Modern Circuits and Systems Technologies (MOCAST2016), Thessaloniki, Greece, 4-6 May 2017.

71. A. Karakostas, D. Ververidis, S. Nikolopoulos, and I. Kompatsiaris, “SpAtiAL: A Sensor based framework to support Affective Learning”, in Proc. 11th 3D True Vision: Research and Applications in Future 3D Media Conference (3DTV-CON 2017), Copenhagen, Denmark, 5-7 June 2017.

72. M. Tits, J. Tilmanne and N. d'Alessandro, “Morphology independent feature engineering in motion capture database for gesture evaluation”, to appear in Proc. 4th International Workshop on Movement and Computing (MOCO’17), London, UK, 28-30 June 2017.

73. A. Grammatikopoulou, S. Laraba, O. Sahbenderoglu, K. Dimitropoulos and N. Grammalidis, “An Adaptive Framework for the creation of body-motion-based games”,to appear in 9th International Conference on Virtual Worlds and Games



for Serious Applications (VS-Games 2017), Athens, Greece, 6-8 September 2017.

4.3 Exploitation In this section, we briefly present the strategy developed for the exploitation of i-Treasures outcomes. More details can be found in D8.8 “Market analysis and exploitation strategy” and D1.2 “Technology implementation plan”.

The exploitable outcomes, as identified by partners, are shown in Table 4. The list includes software tools, methodologies and datasets for ICH analysis and education. The outcomes were efficiently grouped into six categories (see D8.8, Section 3):

• Game-like applications for ICH: Includes the game-like applications for ICH learning developed within i-Treasures for the different ICH sub-use cases.

• Customised games for non-ICH applications: Several tools developed in the i-Treasures project can be combined and modified to develop new exploitable outcomes. A tool for martial arts teaching is a potential exploitable item that can be developed based on our motion capture and educational tools (e.g., the customisable generic game and the pedagogical planner). Similarly, we can create a rehabilitation platform for senior citizens and injured individuals, which could be used to overcome difficulties that occur when an injured individual goes to the hospital for rehabilitation.

• Outcomes of ICH analysis: This category includes software tools (and datasets) developed within WP3, WP4, and WP5 for ICH capture, analysis and visualisation.

• Hardware tools: The wearable hyper-helmet.

• Educational tools: Tools and methodologies for ICH education developed within WP5.

• i-Treasures platform: The i-Treasures Web-platform for research and education, which integrates all the i-Treasures components and functionalities.

Table 4: List of i-Treasures exploitable outcomes in D8.8

No Exploitable result Relevant WP Involved partners

Game-like applications for ICH

1 Game-like application for learning/practicing Tsamiko dance WP5 TT, CERTH

2 Game-like application for learning/practicing Calus dance WP5 TT, CERTH

3 Game-like application for learning/practicing Walloon dance WP5 TT, UMONS

4 Game-like application for learning/practicing Salsa dance WP5 TT, CERTH

5 Game-like application for learning Human BeatBox WP5 TT, CNRS, UPMC, UMONS



6 Game-like application for learning Byzantine singing WP5 TT, UOM, UMONS

7 Game-like application for virtual pottery WP5 TT, CERTH

8 “Pottery master” game-like application WP5 TT

9 Game-like application for contemporary music composition WP5 TT, UOM,

ARMINES, AUTH

Customised games for non-ICH applications

10 Tool for martial arts teaching WP5 TT, UMONS, CERTH

11 Rehabilitation platform WP5 TT, UMONS, CERTH

Outcomes of ICH analysis

12 Body motion analysis algorithms based on Linear Dynamical Systems WP3 CERTH

13 Skeletal motion capture and evaluation module WP3 CERTH

14 “x2Gesture” 3D gesture recognition engine WP3 UOM, ARMINES

15 MyoWeb toolkit for capturing, analysis and visualisation of arm/finger gestures WP4 CERTH

16 MotionMachine toolbox for prototyping of motion feature sets WP3,4 UMONS

17 Software tools for recording and post-processing of vocal tract signals (based on hyper-helmet) WP3 CNRS

18 EEG-based affective state recognition module WP3 AUTH

19 Text to Song (TTS) synthesis module WP5 ACAPELA

20 Singing voice corpora for Byzantine music and Canto a Tenore WP5 ACAPELA

21 Singing voice corpus for English Pop WP5 ACAPELA

22 Intangible Musical Instrument WP4 UOM, ARMINES, AUTH

23 Customisable generic game for dance learning WP5 TT, UMONS, CERTH

Hardware tools

24 Wearable hyper-helmet with multiple embedded sensors for vocal tract signal capturing WP3 UPMC



Educational tools

25 New methodology for ICH education WP5 CNR

26 Pedagogical Planner WP5 CNR

27 e-courses for ICH at the i-Treasures LMS WP5 CNR, CERTH, CNRS, UOM, ARMINES

i-Treasures platform

28 i-Treasures Web-platform for research and education WP1-WP8 All partners

Business model canvasses were prepared for each exploitable outcome, identifying key elements for turning it into a product, including customers, route to market, value proposition, financing, etc. (Figure 48) [83]. Market sectors and competitors for the developed technologies were then identified and market analysis was performed for each sector to identify market growth and trends, as well as opportunities and challenges for exploitable outcomes.

The main opportunities identified were the lack of big players in the ICH industry, the rising popularity of simulation and 3D visualisation applications in market sectors like education and health, and the collaboration chances with relevant projects. The adaptation of our technologies to new ICHs and corresponding user requirements, funding issues on educational tools, and decreasing popularity of PC games caused by the rising popularity of mobile games, are the main challenges for the exploitation of i-Treasures outcomes.

To plan joint and individual exploitation strategies, partners were categorised as industrial and academic. Industrial project partners are focusing their exploitation activities on improving their current operation and business position in existing markets, and on the creation of and preparation for new markets, with the intention to secure a strong leadership position in these new markets. The exploitation goals of academic partners (i.e., universities and research centres) are different, yet complementary, to those of industrial partners. Technical developments will be integrated quickly into the teaching curricula and research agendas of i-Treasures’ partners, giving themselves as well as their graduates a competitive edge.

Moreover, for each exploitable outcome the role of each partner was indentified, based on the following categories: research partner, technology provider, application developer system integrator, service provider. Based on these roles, joint exploitation plans for the future were outlined. In addition, issues related to IPR management and licensing, background, foreground (including joint ownership), transfer of results, granting licenses, protecting results and open source distribution of i-Treasures IP were clarified.

Finally, individual exploitation strategies for each partner were developed (see D8.8).



Figure 48: Consolidated Business Model Canvas (BMC) for the game-like applications

for ICH learning.

.



5. Conclusions ICH represents an important part of the world heritage. However, since it is mainly transmitted orally from generation to generation, there is always a risk that certain elements of ICH could die out or disappear. To this end, ICT technologies can be employed to improve the presentation and transmission of ICH, raise public awareness, provide seamless and universal access to cultural resources, and support services for research and education.

Towards this direction, the i-Treasures project proposed a novel holistic approach for the safeguarding and transmission of ICH, which comprises of the following elements:

• Novel tools for ICH capture and analysis based on multisensory technology.

• Novel methodologies for the extraction of new knowledge that enable researchers to find hidden correlations and give rise to a deeper understanding of specific forms of ICH.

• Advanced semantic analysis, documentation and browsing functionalities for ICH resources.

• Innovative solutions to support learning design and pedagogical planning in the field of ICH education, covering both theoretical and practical aspects of ICH based on innovative tools like the game-like applications or the TTS module.

• Novel sensorimotor learning tools for the transmission of rare know-how from ICH experts to learners.

• Novel intangible musical instrument for contemporary music composition based on natural gesture and emotion recognition that enables new ways of artistic expression.

• Open-source content management system enriched with a significant number of functionalities contributing to the preservation and widespread dissemination of ICH treasures.

The research and development of these tools and technologies presented several challenges, the most important of which were a) the need to design a system that would satisfy the users’ needs and expectations, and b) the effective integration of the different methodologies, tools and sensors into a an open and an extendable platform for research and education.

To address the first challenge, a participatory approach was followed, which demanded the active involvement of the ICH experts in all stages of development, from the definition of requirements, to the design of the educational scenarios and the game interfaces, to the identification of the specificities and the most important elements of each ICH (for the analysis), to the organisation of the demonstration of the final system, to the evaluation of the system and the update of the requirements.

Throughout the life of the project it was made clear that the establishment of close partnerships with ICH practitioners, anthropologists and social scientists in defining what aspects of intangible heritage should be captured and safeguarded was of high importance for the success of this project and, indeed, of every other project in the cultural heritage area. Each heritage expression has different safeguarding needs, which require a different combination of technologies, and other safeguarding measures, which can only be accurately and effectively identified with the help of the experts. Towards this end, ethnographic perspectives should be embraced in the



development of technologies. This can enable a holistic approach towards the conceptualisation and identification of the safeguarding needs of each ICH form.

The second challenge required close collaboration between partners, in order to ensure compatibility of modules and input/outputs and seamless integration, but also close monitoring of technological developments. The development of the game-like applications is a characteristic paradigm of the multiple parameters partners had to consider when designing and developing the different modules. It required not only the integration and communication of various sensors, software modules, third party controller tools, network protocols, into a common framework, but also the development of new algorithms for 3D visualisation and user evaluation. To make things more complicated, in several cases we had to upgrade our modules to support newer and better versions of sensors (e.g., Kinect II instead of Kinect I). In addition, for the implementation of the different game scenarios, partners had to closely cooperate with the experts to take into consideration the specificities of each ICH sub-use case. To overcome these problems, the project focused its efforts on developing a robust common framework that would ensure easy development of future games for other ICH use cases and seamless integration of new software modules and sensors. Similarly, for the development of the Web-platform, partners adopted open source tools and software that ensure easy upgrade and extensibility, while special emphasis was given to the implementation of the communication between the Web-platform and other modules so as to be as simple and robust as possible, again using standard communication protocols and services.

In the final year of the project, the main challenges were user involvement and adoption of the platform. To this end, a clear and solid dissemination plan was established by the Consortium, which, among others, included a) the organisation of local training workshops for ICH safeguarding based on our technologies in three different European cities (Thessaloniki, Mons, Corsica) and b) the introduction of our tools and technologies in the cross-disciplinary MA programme “Educational Sciences - Learning Technologies” at the Aristotle University of Thessaloniki. In both cases, our tools have been successfully used by the participants (that came from the cultural heritage and educational communities) for the development of small safeguarding projects for local ICH expressions in co-operation with local communities or cultural organisations.

This training process was an opportunity for scientific experimentation and for expanding and challenging peoples’ understanding about not only what intangible heritage is, but also how it can be safeguarded. It became clear, that as intangible heritage is constantly changing in response to nature and its broader socio-political context, so must heritage professionals and communities adapt their safeguarding tools. To this end, the i-Treasures platform can play an important role as an educational resource in support of ICH transmission. By creating new knowledge about cultural expressions, it can have a beneficial impact in different learning environments, such as museums, cultural centres or schools. Although it cannot replace the physical intergenerational contact that is at the heart of intangible heritage transmission, it, nevertheless, is a mechanism for raising awareness on the safeguarding of cultural expressions. More importantly, it provides a new methodology and tools for conducting research and creating educational programmes.

In addition to this training initiative, numerous activities have been organised in cooperation with our ICH experts and local cultural/educational organisations for the demonstration of our technologies in the context of 10 ICH sub-use cases, involving more than 650 users of different ages and national/educational/cultural background (school students, ICH practitioners and apprentices, general public) in various scenarios of use (courses taught in schools, online courses for ICH beginners,



demonstration sessions in science festivals for the general public) and in different premises. The users were able not only to experience the Web-platform as ICH learners or experts but also to use the developed technologies to create their own ICH-related tools and content.

User feedback suggests that the developed technologies constitute an innovative and promising set of tools that respect ICH tradition and have the potential to significantly contribute to its safeguarding and transmission, by taking advantage of recent advances in ICT technologies to analyse ICH content in a deeper and more effective way and to boost ICH education in ways that make it more appealing and attractive to younger generations. The users also pointed out issues that could be further explored and enhanced, e.g., the display of the performance evaluation results, but they also proposed several interesting ideas about how our technologies could be improved or have a greater impact.

Although the feedback was in general very positive, there were two issues that we should further take into consideration when designing systems for ICH research and education, like the i-Treasures Web-platform: fear that new technologies may negatively affect the authenticity of an ICH and limited familiarity with computers.

More specifically, while the vast majority of users felt that the use of new technologies greatly benefits ICH safeguarding, users and experts that live in small local communities were, initially at least, more cautious or worried or even negative about our tools and how they could potentially affect the authenticity of their ICH or, in the case of transmission, whether these tools were an attempt to push aside traditional ways of learning and transmission and substitute human connection/interaction with man-machine interfaces. Again, the role of the experts was crucial in introducing and explaining the potential of our technologies to local people and convincing them to use them: first, by highlighting the aspect of capture and analysis, which allows for the better understanding and documentation of ICH, and secondly, by underlining the advantages of modern tools for ICH education that are more appealing to younger people and also attract possible interest from around the world, making their culture known to and appreciated by a much wider audience, which would not be reached otherwise. If the experts incorporate new technologies in their everyday practice, then they will much more easily convince other practitioners to accept and use them as well. Also, since ICH is closely related to local communities and their way of life, it is important to not isolate the developed tools from this reality. Ideally, they should be used within a community along with more traditional methods for education and transmission.

Basic computer literacy is an important pre-requisite for effectively using the developed tools. The lack of experience in the use of ICT technologies created problems for some users, especially older people. Younger people and, especially, children, who are very familiar with such technologies and use them in their everyday life, were more receptive towards our tools and used them in a more effective way; especially educational tools like the LMS courses and game-like applications that provide a modern and enjoyable way to get acquainted with an ICH, more suitable for younger people. Training the ICH experts and the users is extremely important in order to persuade them to use similar tools for ICH transmission.

Overall, feedback from users, both experts and apprentices, from various ICH domains as well as technical assessment of the system by ICT experts, showed the great potential of ICT technologies in capturing, analysing, modelling, documenting and transmitting ICH and established that the i-Treasures Web-platform constitutes a very positive and important step towards the adoption of such technologies for ICH transmission and safeguarding. Although, such a resource cannot replace human interaction, it nevertheless offers an opportunity for different modes of transmission



and engagement with intangible heritage. Moreover, it raises awareness about the value and importance of such cultural expressions beyond the communities directly concerned. Future advances on sensor technologies, both in terms of accuracy and cost-effectiveness, are expected to further boost the use of ICT technologies in the preservation of a plethora of ICH domains.



6. References [1] UNESCO: http://www.unesco.org/culture/ich/index.php?lg=en&pg=00002

[2] K. Dimitropoulos, S. Manitsaris, F. Tsalakanidou, S. Nikolopoulos, B. Denby, S. Al Kork, L. Crevier-Buchman, C. Pillot-Loiseau, S. Dupont, J. Tilmanne, M. Ott, M. Alivizatou, E. Yilmaz, L. Hadjileontiadis, V. Charisis, O. Deroo, A. Manitsaris, I. Kompatsiaris, and N. Grammalidis, “Capturing the Intangible: An Introduction to the i-Treasures Project”, in Proc. 9th International Conference on Computer Vision Theory and Applications (VISAPP2014), Lisbon, Portugal, 5-8 January 2014.

[3] K. Dimitropoulοs, S. Manitsaris, F. Tsalakanidou, B. Denby, L. Crevier-Buchman, S. Dupont, S. Nikolopoulos, Y. Kompatsiaris, V. Charisis, L. Hadjileontiadis, F. Pozzi, M. Cotescu, S. Ciftsi, A. Katos, A. Manitsaris and N. Grammalidis, “A multimodal approach for the safeguarding and transmission of Intangible Cultural Heritage: The case of i-Treasures”, IEEE Intelligent Systems, 2017, submitted for publication.

[4] F. Pozzi, M. Alivizatou, F. M. Dagnino, and M. Ott, “Going Beyond Preservation: How to support Technology-Enhanced Learning in ICH education”, International Journal of Heritage in the Digital Era, Vol. 4, No 1, pp. 21-40, March 2015.

[5] M. Alivizatou-Barakou, A. Kitsikidis, F. Tsalakanidou, K. Dimitropoulos, G. Chantas, S. Nikolopoulos, S. Al Kork, B. Denby, L. Buchman, M. Adda-Decker, C. Pillot-Loiseau, J. Tillmane, S. Dupont, B. Picart, F. Pozzi, M. Ott, E. Yilmaz, V. Charisis, S. Hadjidimitriou, L. Hadjileontiadis, M.Cotescu, C.Volioti, A. Manitsaris, S. Manitsaris, and N. Grammalidis, “Intangible Cultural Heritage and new technologies: Challenges and opportunities for cultural preservation and development”, in M. Ioannides, N. Magnenat-Thalmann, and G. Papagiannakis (Eds), Mixed Reality and Gamification for Cultural Heritage, Springer International Publishing, February 2017.

[6] F. Pozzi, A. Antonaci, F. M. Dagnino, M. Ott and M. Tavella, “A Participatory Approach to Define User Requirements of a Platform for Intangible Cultural Heritage Education”, in Proc. 9th International Conference on Computer Vision Theory and Applications (VISAPP2014), Lisbon, Portugal, 5-8 January 2014.

[7] F. M. Dagnino, L. Hadjileontiadis, M. Ott, and F. Pozzi, “An integrated platform supporting Intangible Cultural Heritage learning and transmission: definition of requirements and evaluation criteria”, Journal of Computing and Information Technology (CIT), Vol. 22, No 4, pp. 277–292, 2014.

[8] M. Ott, F. M. Dagnino, F. Pozzi, and M. Tavella, "Widening Access to Intangible Cultural Heritage: Towards the Development of an Innovative Platform", in Proc. HCI International 2014, Heraklion, Greece, 22-27 June 2014.

[9] F. Pozzi, F.M. Dagnino, M. Ott and A. Antonaci “i-Treasures and Intangible Cultural Heritage Education”, in Proc. Museums and the Web Florence International Conference (MWF2014), Pre-conference workshop, Florence, Italy, 18-21 February 2014.

[10] A. Antonaci, P. Bravi, F. Dagnino, M. Lutzu, M. Ott, S. Pilosu, F. Pozzi, “Exploring the “Intangible” through ICT”, in Proc. 6th International Conference of Education, Research and Innovation (ICERI2013), Seville, Spain, 18-20 November 2013.

[11] A. Antonaci, F. Dagnino, M. Ott, F. Pozzi, P. Bravi, and M. Lutzu, “Digital technology and transmission of Intangible Cultural Heritage: the case of Cantu a



Tenore”, in Digital Heritage 2013, Marseille, France, 28 October 28 – 1 November 2013.

[12] Microsoft Kinect sensor: http://www.microsoft.com/en-us/kinectforwindows/

[13] Leap Motion sensor: https://www.leapmotion.com/

[14] A. Kitsikidis, K. Dimitropoulos, S. Douka, and N. Grammalidis, “Dance analysis using multiple Kinect sensors”, in Proc. 9th International Conference on Computer Vision Theory and Applications (VISAPP2014), Lisbon, Portugal, 5-8 January 2014.

[15] A. Kitsikidis, K. Dimitropoulos, E. Yilmaz, S. Douka and N. Grammalidis, "Multi-sensor technology and fuzzy logic for dancer’s motion analysis and performance evaluation within a 3D virtual environment", in Proc. HCI International 2014, Heraklion, Greece, 22-27 June 2014.

[16] A. Kitsikidis, N. V. Boulgouris, K. Dimitropoulos, and N. Grammalidis, "Unsupervised dance motion patterns classification from fused skeletal data using exemplar-based HMMs", International Journal of Heritage in the Digital Era, Vol. 4, No. 2, pp. 209-220, June 2015.

[17] K. Dimitropoulos, A. Kitsikidis, and N. Grammalidis, "Information technologies in traditional dance analysis", Journal of Science of Dance, April-May 2014

[18] K. Dimitropoulos, A. Kitsikidis P. Barmpoutis, and N. Grammalidis, “Classification of multidimensional time-evolving data using histograms of Grassmannian points”, to appear in IEEE Transactions on Circuits and Systems for Video Technology, 2017.

[19] K. Dimitropoulos, P. Barmpoutis, A. Kitsikidis, and N. Grammalidis, “Extracting dynamics from multi-dimensional time-evolving data using a bag of higher-order Linear Dynamical Systems”, in Proc. 11th International Conference on Computer Vision Theory and Applications (VISAPP 2016), Rome, Italy, 27-29 February 2016.

[20] T. Ravet, J. Tilmanne, and N. D'Alessandro, "Hidden Markov Model based real-time motion recognition and following", in Proc. International Workshop on Movement and Computing (MOCO'14), Paris, France, 16-17 June 2014.

[21] J. Tilmanne, N. D'Alessandro, T. Ravet, M. Astrinaki, and A. Moinet, "Full-body gait reconstruction using covariance-based mapping within a realtime HMM-based framework", in Proc. 50th Annual Convention of the AISB (AISB-50), pp. 313-316, London, UK, 1 - 4 April 2014.

[22] J. Tilmanne, N. d’Alessandro, M. Astrinaki and T. Ravet, “Exploration of a Stylistic Motion Space Through Realtime Synthesis”, in Proc. 9th International Conference on Computer Vision Theory and Applications (VISAPP2014), Lisbon, Portugal, 5-8 January 2014.

[23] N. d’Alessandro, J. Tilmanne, M. Astrinaki, T. Hueber, R. Dall, T. Ravet, A. Moinet, H. Cakmak, O. Babacan, A. Barbulescu, V. Parfait, V. Huguenin, E. Sumeyye Kalayci, and Qiong Hu, “Reactive Statistical Mapping: Towards the Sketching of Performative Control with Data”, in Proc. 9th International Summer Workshop on Multimodal Interfaces (eNTERFACE'13), Lisbon, Portugal, July 15th- August 9th 2013.

[24] S. Laraba, J. Tilmanne, and T. Dutoit, “Adaptation procedure for HMM-based sensor-dependent gesture recognition”, in Proc. 8th ACM SIGGRAPH Conference on Motion in Games (MIG2015), Paris, France, 16-18 November 2015.



[25] S. Laraba and J. Tilmanne. “Dance performance evaluation using Hidden Markov Models”, Computer Animation and Virtual Worlds Journal (CASA), Vol. 27, No. 3-4, pp. 321–329, May-August 2016.

[26] C. Volioti, S Manitsaris, E. Katsouli, A. Manitsaris, “x2Gesture: how machines could learn expressive gesture variations of expert musicians”, in Proc. 16th International Conference on New Interfaces for Musical Expression (NIME2016), Brisbane, Australia, 11-15 July 2016.

[27] S. Manitsaris, A. Glushkova, E. Katsouli, A. Manitsaris and C. Volioti, "Modelling gestural know-how in pottery based on state-space estimation and system dynamic simulation", in Proc. 6th International Conference on Applied Human Factors and Ergonomics (AHFE 2015), Las Vegas, USA, 26-30 July 2015.

[28] C. Volioti, S. Manitsaris, and A. Manitsaris, "Offline statistical analysis of gestural skills in pottery interaction", in Proc. International Workshop on Movement and Computing (MOCO'14), Paris, France, 16-17 June 2014.

[29] O. Bouzos, Y. Jacob, S. Manitsaris and A. Glushkova, “3D-scene modelling of professional gestures when interacting with moving, deformable and revolving objects”, in Proc. 3rd International Workshop on Movement and Computing (MOCO’16), Thessaloniki, Greece, 5-6 July 2016.

[30] pmd[vision]® CamBoard nano: http://www.pmdtec.com/products_services/reference_design.php

[31] Emotiv Inc. (2014). Emotiv EPOC & Testbench specifications [Online]. Available: https://emotiv.com/product-specs/Emotiv%20EPOC%20Specifications%202014.pdf

[32] S. Hadjidimitriou, V. Charisis and L. Hadjileontiadis, “Towards a practical subject-independent affective state recognition based on time-domain EEG feature extraction”, International Journal of Heritage in the Digital Era, Vol. 4, No. 2, pp. 165-177, June 2015.

[33] A. Jaumard-Hakoun, S. K. Al Kork, M. Adda-Decker, A. Amelot, L. Buchman, T. Fux, C. Pillot, P. Roussel, M. Stone, G. Dreyfus, and B. Denby, “Capturing, Analyzing, and Transmitting Intangible Cultural Heritage with the i-Treasures Project”, in Proc. Ultrafest VI, Edinburgh, UK, 6-8 November 2013.

[34] S. K. Al Kork, A. Jaumard-Hakoun, M. Adda-Decker, A. Amelot, L. Buchman, T. Fux, C. Pillot, P. Roussel, M. Stone, G. Dreyfus, and B. Denby, "A multi-sensor hyper-helmet to capture rare singing, an Intangible Cultural Heritage study", in Proc. 10th International Seminar on Speech Production (ISSP2014), Cologne, Germany, 5 - 8 May 2014.

[35] P. Chawah, S. K. Al Kork, T. Fux, M. Adda-Decker, A. Amelot, N. Audibert, B. Denby, G. Dreyfus, A. Jaumard-Hakoun, C. Pillot-Loiseau, P. Roussel, M. Stone, K. Xu, and L. Buchman, "An educational platform to capture, visualize and analyze rare singing", in Proc. 15th Annual Conference of the International Speech Communication Association (Interspeech 2014), Singapore, 14-18 September 2014.

[36] L. Crevier-Buchman, A. Amelot, S. K. Al Kork, M. Adda-Decker, N. Audibert, P. Chawah, B. Denby, T. Fux, A. Jaumard-Hakoun, P. Roussel, M. Stone, J. Vaissiere, K. Xu, and C. Pillot-Loiseau, “Acoustic data analysis from multi-sensor capture in rare singing: Cantu in Paghjella case study”, International Journal of Heritage in the Digital Era, Vol. 4, No. 1, pp. 120-132, 2015.



[37] T. Fux, A. Amelot, L. Crevier-Buchman, C. Pillot-Loiseau, and M. Adda-Decker, "On the use of accelerometer sensors to study nasality in speech and singing voice", in Proc. 10th International Seminar on Speech Production (ISSP2014), Cologne, Germany, 5 - 8 May 2014.

[38] L. Crevier-Buchman, T. Fux, C. Pillot-Loiseau, A. Amelot, and M. Adda-Decker, "Measuring vocal tract agility from videofiberscopic data: The Human BeatBox example", in Proc. The Voice Foundation 2014 - 43rd Annual Symposium: Care of the Professional Voice, Philadelphia, USA, 28 May - 1 June 2014.

[39] A. Jaumard-Hakoun, K. Xu, P. Roussel-Ragot, G. Dreyfus, M. Stone and B. Denby, “Tongue contour extraction from ultrasound images based on Deep Neural Network”, in Proc. 18th International Congress on Phonetic Sciences (ICPhS 2015), Glasgow, UK, 10-14 August 2015

[40] K. Xu, Y. Yang, A. Jaumard-Hakoun, C. Leboullenger, G. Dreyfus, P. Roussel, M. Stone and B. Denby, “Robust contour tracking in ultrasound tongue image sequences”, Clinical Linguistics & Phonetics, Vol. 30, No. 3-5, pp. 313-327, March-May 2016.

[41] K. Xu, T. Gábor Csapó, P. Roussel, and B. Denby, “A comparative study on the contour tracking algorithms in ultrasound tongue images with automatic re-initialization”, Journal of the Acoustical Society of America, Vol. 139, No. 5, pp. 154-160, May 2016.

[42] Kele Xu, Yin Yang, A. Jaumard-Hakoun, M. Adda-Decker, A. Amelot, S. K. Al Kork, L. Crevier-Buchman, P. Chawah, G. Dreyfus, T. Fux, C. Pillot-Loiseau, P. Roussel, M. Stone, and B. Denby, "3D tongue motion visualization based on ultrasound image sequences", in Proc. 15th Annual Conference of the International Speech Communication Association (Interspeech 2014), Singapore, 14-18 September 2014.

[43] K. Xu, Y. Yang, A. Jaumard-Hakoun, C. Leboullenger, G. Dreyfus, P. Roussel, M. Stone and B. Denby, “Development of a 3D tongue motion visualization platform based on ultrasound image sequences”, in Proc. 18th International Congress on Phonetic Sciences (ICPhS 2015), Glasgow, UK, 10-14 August 2015.

[44] K. Xu, Y. Yang, C. Leboullenger, P. Roussel, and B. Denby, “Contour-based 3D tongue motion visualization using the ultrasound image sequences”, in Proc. 41st IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2016), Shanghai, China, 20-25 March 2016.

[45] Kele Xu, P. Roussel, T. G. Csapó, and B. Denby, “Convolutional neural network-based automatic classification of midsagittal tongue gestural targets using B-mode ultrasound images”, Journal of the Acoustical Society of America, Vol. 141, No. 6, June 2017.

[46] B. Picart, S. Brognaux, and S. Dupont, "Analysis and automatic recognition of human beatbox sounds: A comparative study", in Proc. 40th IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2015), Brisbane, Australia, 19-24 April 2015.

[47] N. Grammalidis, K. Dimitropoulos, F. Tsalakanidou, A. Kitsikidis, P. Roussel, B. Denby, P. Chawah, L. Buchman, S. Dupont, S. Laraba, B. Picart, M. Tits, J. Tillmane, S. Hadjidimitriou, L. Hadjileontiadis, V. Charisis, C. Volioti, A. Stergiaki, A. Manitsaris, O. Bouzos, and S. Manitsaris, “The i-Treasures Intangible Cultural Heritage dataset”, in Proc. 3rd International Workshop on Movement and Computing (MOCO’16), Thessaloniki, Greece, 5-6 July 2016.



[48] G. Chantas, S. Nikolopoulos, and I. Kompatsiaris, “Multi-entity Bayesian Networks for Treasuring the Intangible Cultural Heritage”, in Proc. 9th International Conference on Computer Vision Theory and Applications (VISAPP2014), Lisbon, Portugal, 5-8 January 2014.

[49] G. Chantas, A. Kitsikidis, S. Nikolopoulos, K. Dimitropoulos, S. Douka, I. Kompatsiaris, and N. Grammalidis, "Multi-entity Bayesian networks for knowledge-driven analysis of ICH content", in Proc. 1st International Workshop on Computer vision + ONTology Applied Cross-disciplinary Technologies ( ) - 13th European Conference on Computer Vision (ECCV 2014), Zurich, Switzerland, 7 September 2014.

[50] C. Volioti, S. Manitsaris, E. Hemery, S. Hadjidimitriou, V. Charisis, L. Hadjileontiadis, E. Katsouli, F. Moutarde and A. Manitsaris, “A natural user interface for gestural expression and emotional elicitation to access the musical Intangible Cultural Heritage”, to appear in Journal on Computing and Cultural Heritage (JOCCH), ACM, 2017.

[51] E. Hemery, S. Manitsaris, and F. Moutarde, “A tabletop instrument for manipulation of sound morphologies with hands, fingertips and upper-body”, in Proc. 3rd International Workshop on Movement and Computing (MOCO’16), Thessaloniki, Greece, 5-6 July 2016.

[52] C. Volioti, S. Hadjidimitriou, S. Manitsaris, L. Hadjileontiadis, V. Charisis, and A. Manitsaris, “On mapping emotional states and implicit gestures to sonification output from the “Intangible Musical Instrument””, in Proc. 3rd International Workshop on Movement and Computing (MOCO’16), Thessaloniki, Greece, 5-6 July 2016.

[53] E. Hemery, S. Manitsaris, F. Moutarde, C. Volioti, and A. Manitsaris, “Towards the design of a natural user interface for performing and learning musical gestures”, in Proc. 6th International Conference on Applied Human Factors and Ergonomics (AHFE 2015), 26-30 July 2015, Las Vegas, USA.

[54] L. J. Hadjileontiadis, “Conceptual blending in biomusic composition space: the “Brainswarm” paradigm”, in Proc. Joint 11th Sound and Music Computing Conference and 40th International Computer Music Conference, Athens, Greece, 14 -20 September 2014.

[55] J. Tilmanne and N. d’Alessandro, “MotionMachine: A new framework for motion capture signal feature prototyping”, in Proc. 2015 European Signal Processing Conference (EUSIPCO 2015), Nice, France, 31 August – 4 September 2015.

[56] T. Ravet, N. d’Alessandro, J. Tilmanne, and S. Laraba, “Motion Data and Machine Learning: Prototyping and Evaluation”, in Proc. ACM CHI Conference on Human Factors in Computing Systems 2016 (ACM CHI 2016), San Jose, USA, 7-12 May 2016.

[57] M. Tits, J. Tilmanne, and N. d'Alessandro, “A novel tool for motion capture database factor statistical exploration”, in Proc. 3rd International Workshop on Movement and Computing (MOCO’16), Thessaloniki, Greece, 5-6 July 2016.

[58] M. Tits, J. Tilmanne and N. d'Alessandro, “Morphology independent feature engineering in motion capture database for gesture evaluation”, to appear in Proc. 4th International Workshop on Movement and Computing (MOCO’17), London, UK, 28-30 June 2017.

[59] D. Ververidis, S. Karavarsamis, S. Nikolopoulos, and I. Kompatsiaris, “Pottery gestures style comparison by exploiting MYO sensor and forearm anatomy”, in



Proc. 3rd International Workshop on Movement and Computing (MOCO’16), Thessaloniki, Greece, 5-6 July 2016.

[60] A. Karakostas, D. Ververidis, S. Nikolopoulos, and I. Kompatsiaris, “SpAtiAL: A Sensor based framework to support Affective Learning”, in Proc. 11th 3D True Vision: Research and Applications in Future 3D Media Conference (3DTV-CON 2017), Copenhagen, Denmark, 5-7 June 2017.

[61] F. M. Dagnino, M. Ott, and F. Pozzi, “Addressing key challenges in Intangible Cultural Heritage education”, International Journal of Heritage in the Digital Era, Vol. 4, No. 2, pp. 193-207, June 2015.

[62] F. M. Dagnino and F. Pozzi, “The i-Treasures project: capturing the intangible through Information and Communication Technologies”, in Marta Severo and Séverine Cachat (Eds), Patrimoine culturel immatériel et numérique: transmission, participation, enjeux, L’Harmattan, September 2016.

[63] F. Pozzi, A. Ceregini, F. M. Dagnino, M. Ott and M. Tavella “Closing the "learning design lifecycle" with the Pedagogical Planner”, Special issue of European Journal of Open, Distance and E-Learning (EURODL), pp. 103-116, December 2016.

[64] F. Pozzi, A. Ceregini, F. M. Dagnino, M. Ott and M. Tavella, “Supporting the ‘whole learning design life-cycle’ through the pedagogical planner”, in Proc. 1st D4Learning International Conference Innovations with Digital Learning for Inclusion (D4L), pp. 90-97, Aalborg, Denmark, 17-20 November 2015.

[65] M. Ott, F. M. Dagnino, and F. Pozzi, “Intangible Cultural Heritage: Towards Collaborative Planning of Educational Interventions”, Computers in Human Computers in Human Behaviour, Vol. 51, Part B, pp. 1314–1319, October 2015.

[66] E. Yilmaz, D. Uğurca, C. Şahin, F. M. Dagnino, M. Ott, F. Pozzi, K. Dimitropoulos, F. Tsalakanidou, A. Kitsikidis, S. K. Al Kork, K. Xu, B. Denby, P. Roussel, P. Chawah, L. Buchman, M. Adda-Decker, S. Dupont, B. Picart, J. Tilmanne, M. Alivizatou, L. Hadjileontiadis, V. Charisis, A. Glushkova, C. Volioti, A. Manitsaris, E. Hemery, F. Moutarde, and N. Grammalidis, “Novel 3D game-like applications driven by body interactions for learning specific forms of Intangible Cultural Heritage”, in Proc. 10th International Conference on Computer Vision Theory and Applications (VISAPP2015), Berlin, Germany, 11-14 March 2015.

[67] N. Grammalidis, K. Dimitropoulos, A. Kitsikidis, M. Politou, E. Yilmaz, D. Ugurca, C. Sahin, F.M. Dagnino, M. Ott, F. Pozzi, S. K. Al Kork, K. Xu, B. Denby, P. Roussel, P. Chawah, L. Buchman, M. Adda-Decker, S. Dupont, B. Picart, J. Tilmanne, M. Alivizatou, L. Hadjileontiadis, V. Charisis, S. Hadjidimitriou, A. Glushkova, C. Violioti, A. Manitsaris, E. Hemery, and F. Moutarde, “Novel 3-D game applications for Intangible Cultural Heritage”, in Proc. Pan-Hellenic Conference on Digital Cultural Heritage 2015, Volos, Greece, 24-26 September 2015.

[68] F. M. Dagnino, M. Ott, F. Pozzi, and E. Yilmaz, "Serious Games design: reflections from an experience in the field of Intangible Cultural Heritage education", in Proc. 10th E-learning and Software for Education Conference (eLSE), Bucharest, Romania, 23-24 April 2015.

[69] F. M. Dagnino, M. Ott, F. Pozzi, E. Yilmaz, K. Dimitropoulos, N. Grammalidis, and F. Tsalakanidou, “Designing Serious Games for ICH Education”, in Proc. Digital Heritage Conference 2015 (DH’15), Granada, Spain, 28 September-2 October 2015.



[70] F. M. Dagnino, M. Ott, F. Pozzi, E. Yilmaz, F. Tsalakanidou, K. Dimitropoulos, and N. Grammalidis, "Serious games to support learning of rare ‘intangible’ cultural expressions", in Proc. 9th International Technology, Education and Development Conference (INTED2015), pp. 7184-7194, Madrid, Spain, 2-4 March 2015.

[71] A. Kitsikidis, K. Dimitropoulos, D. Uğurca, C. Baycay, E. Yilmaz, F. Tsalakanidou, S. Douka, and N. Grammalidis, "A game-like application for dance learning using a natural human computer interface", in Proc. HCI International 2015, Los Angeles, USA, 2-7 August 2015.

[72] S. K. Al Kork, D. Uğurca, C. Şahin, P. Chawah, L. Buchman, M. Adda-Decker , K. Xu, B. Denby, P. Roussel, B. Picart, S. Dupont, F. Tsalakanidou, A. Kitsikidis, F. Dagnino, M. Ott, F. Pozzi, M. Stone, and E. Yilmaz, “A novel human interaction game-like application to learn, perform and evaluate modern contemporary singing: “Human Beat Box””, in Proc. 10th International Conference on Computer Vision Theory and Applications (VISAPP2015), Berlin, Germany, 11-14 March 2015.

[73] K. Kokkinidis, A. Stergiaki and A. Tsagaris, “Error proving and sensorimotor feedback for singing voice”, in Proc. 3rd International Workshop on Movement and Computing (MOCO’16), Thessaloniki, Greece, 5-6 July 2016.

[74] V. Charisis, S. Hadjidimitriou, L. Hadjileontiadis, D. Ugurca, and E. Yilmaz, "EmoActivity - An EEG-based gamified emotion HCI for augmented artistic expression: The i-Treasures paradigm", in Proc. HCI International 2015, Los Angeles, USA, 2-7 August 2015.

[75] C.Volioti, E. Hemery, S. Manitsaris, V. Tsekouropoulou, E. Yilmaz, F. Moutarde, and A. Manitsaris, “Music gestural skills development engaging teachers, learners and expert performers”, in Proc. 6th International Conference on Applied Human Factors and Ergonomics (AHFE 2015), 26-30 July 2015, Las Vegas, USA.

[76] A. Grammatikopoulou, S. Laraba, O. Sahbenderoglu, K. Dimitropoulos and N. Grammalidis, “An Adaptive Framework for the creation of body-motion-based games”, 9th International Conference on Virtual Worlds and Games for Serious Applications (VS-Games 2017), Athens, Greece, 6-8 September 2017.

[77] M. Cotescu, “Optimal unit stitching in a unit selection singing synthesis system”, in Proc. 17th Annual Conference of the International Speech Communication Association (InterSpeech2016), San Francisco, USA, 8-12 September 2016, pp. 1255-1259.

[78] A. Glushkova, E. Katsouli, G. Kourvoulis, A. Manitsaris, and C. Volioti, "A Hybrid Content-Learning Management system for education and access to Intangible Cultural Heritage", in Proc. of the 17th International Conference on Computer Supported Education (CSEDU 2015), Lisbon, Portugal, 23-25 May 2015.

[79] P. Kritopoulou, M. Lazaridou, S. Manitsaris, A. Glushkova, C.Volioti, E. Katsouli, and A. Manitsaris, “A sensorimotor e-learning platform for the transmission of technical and artistic motor skills using motion capture”, Journal of Computer Assisted Learning, 2017, submitted for publication.

[80] Europeana Semantic Element Set: http://pro.europeana.eu/page/ese-documentation.

[81] B.M. Byrne, “Structural equation modeling with AMOS: Basic concepts, Applications, and Programming” (2nd ed.), Routledge, London, 2010.



[82] J.F. Hair, R.E. Anderson, R.L. Tatham, and W.C. Black, “Multivariate Data Analysis” (7th ed.), Prentice Hall Publisher, Upper Saddle River, New Jersey, 2008.

[83] Business Model Canvas: http://www.alexandercowan.com/business-model-canvas-templates/

d1.3 i-treasures final report final - cordis...d1.3 i-treasures final report i-treasures ict-600676...

Documents