analyzing and ranking multimedia ontologies for their reuse
Post on 06-May-2015
1.774 Views
Preview:
TRANSCRIPT
Date: 11/04/23Speaker: Ghislain Auguste Atemezing
ANALYZING AND RANKING MULTIMEDIA ONTOLOGIES FOR THEIR REUSE
Master ThesisMáster de investigación en inteligencia artificial
Author: Ghislain Auguste Atemezing
Supervisor: Dr. María del Carmen Suárez de Figueroa Baonza
2Analyzing and Ranking Multimedia Ontologies for their Reuse
Outline
• Introduction
• State of the Art on MultiMedia Ontologies
• Searching MM Ontologies
• Assessing MM Ontologies
• Selecting MM Ontologies
• Conclusions
Introduction (I)
3
• Multimedia content is ubiquitous (Web, TV news, Film, Phone, etc.), and store huge collection of data (Library, Museum, Archives, etc.)
• Multimedia includes a combination of text, audio, still images, animation, video, and interactivity content forms [Chapman 09]
• Many of these contents are available online
Jenny Chapman and Nigel Chapman. Digital Multimedia . John Niley & Sons Ltd, 2009.
Analyzing and Ranking Multimedia Ontologies for their Reuse
Introduction (II)
4Analyzing and Ranking Multimedia Ontologies for their Reuse
• Continuously consuming multimedia contents of different formats and from different sources in web environment (e.g., Google, Flickr, Picassa, Youtube).
• How to efficiently retrieve multimedia objects for web developers and ordinary users?
Introduction (III)
5Analyzing and Ranking Multimedia Ontologies for their Reuse
• Descriptors based on the automatic analysis of audiovisual content are far from what users require.
• Need for correct semantic annotation and representation of multimedia content.
• Recent research focus on the reduction of semantic and conceptual gap between user and machine. That is, based on the content of high-level descriptions. Reusing KNOWLEDGE in ontology engineering.
Introduction (IV)
• Many standards to describe MM content: MPEG-4, MPEG7, IPTC, etc.
• Standards provide descriptors schemas for low level description.
6Analyzing and Ranking Multimedia Ontologies for their Reuse
Introduction (V)
• “Semantic gap”: mismatch between the information that can be extracted from audio-visual data and the interpretation that each user makes in a given situation for the same data [Smeulders 00].
• Many initiatives in the last decade to bridge the gap: MPEG 7 transformations [Hunter 01, Celma 05]; COMM [Arndt07] by creating ontologies for multimedia.
• Methodologies for ontology engineering: METHONDOLOGY, On-To-Knowledge, DILIGENT, and recently NeOn Methodology.
7
A. Smeulders, M. Worring. Content-based image retrieval at the end of the early years. IEEE Trans. Pattern Anal. Mach. Intell., 22:1349–1380, December 2000.
Jane Hunter. Adding Multimedia to the Semantic Web - Building an MPEG-7 Ontology. In International Semantic Web Working Symposium (SWWS), Stanford, 2001.
R. Arnd R. Troncy. COMM: Designing a Well-Founded Multimedia Ontology for the Web. In 6th International Semantic Web Conference ISWC2007, Busan, Korea. Springer, 2007.
Analyzing and Ranking Multimedia Ontologies for their Reuse
Introduction (VI)
8
Main objective: To search, find, analyze, rank and select suitable multimedia (MM) ontologies to be reused in the development of a multimedia ontology called M3 (Multimedia-Multidominio-Multilingüe)
Goal 1: To obtain a rank of MM ontologies to select the most appropriate ones that will be reused in the development of the M3 ontology.
Goal 2: To describe in detail and in a pedagogic way an example of how to apply the methodological guidelines for reusing ontologies in the multimedia domain.
Analyzing and Ranking Multimedia Ontologies for their Reuse
9
Introduction (VII)
•Apply and extend Neon Methodology guidelines [Suárez-Figueroa, 2010] for reusing domain ontology in MM:
•domain ontology search: look for candidate domain ontologies that could satisfy the needs of the M3 Ontology.
•domain ontology assessment: find out if the set of candidate domain ontologies are useful for the development of the M3 Ontology.
•domain ontology selection: find out which domain ontologies are the most suitable for the development of the M3 Ontology.
•domain ontology integration: integrate the domain ontologies selected in the M3 Ontology.
General
Process
M.C. Suárez-Figueroa. PhD Thesis: NeOn Methodology for Building Ontology Networks: Specification, Scheduling and Reuse. España. Universidad Politécnica de Madrid. Junio 2010.
Analyzing and Ranking Multimedia Ontologies for their Reuse
10Analyzing and Ranking Multimedia Ontologies for their Reuse
Outline
• Introduction
• State of the Art on MultiMedia Ontologies
• Searching MM Ontologies
• Assessing MM Ontologies
• Selecting MM Ontologies
• Conclusions
11Analyzing and Ranking Multimedia Ontologies for their Reuse
Outline
• Introduction
• State of the Art on MultiMedia Ontologies
• MPEG-7
• Ontologies describing MM objects
• Ontologies describing Shapes and Images
• Ontologies describing Visual Resource Object
• Ontologies describing Audio and Music
• Application Ontologies
12Analyzing and Ranking Multimedia Ontologies for their Reuse
MPEG 7 Standard: “Multimedia Content Description”
Descriptors Components
Visual Features Color, Texture, Shape, Motion, Localization, Face recognition.
Color Descriptors Color space, ColorQuantization, Dominant Colors, Scalable Color, Color Layout, Color-Structure, GoF/GoP Color.
Texture Descriptors Homogeneous Texture, Edge Histogram, Texture Browsing
Shape Descriptors Region Shape, Contour Shape, Shape 3D
Motion Descriptors Camera Motion, Motion Trajectory, Parametric Motion, Motion Activity
Localization Descriptors Region locator, Spatio-temporal locator
Audio Framework Basic (AudioWaveform, AudioPower), Basic Spectral, Timbral Temporal and Timbral Spectral
Ontologies for describing MM objects
13
•It is composed of multimedia patterns specializing the DOLCE design patterns for Descriptions & Situations and Information Objects, [Arndt et al., 07], OWL DL.•Scope covered: Multimedia, audio/music, image.•Ontological Resource reused: DOLCE, DnS, IO. •Non Ontological Resource reused: MPEG 7
COMM
•It is targeted for rich presentations in the web like SMIL, SVG and Flash.,[C. Scherp , A.Saathoff 10], OWL Full.•Scope covered: Multimedia, audio/music, image, videoOntological Resource reused: DOLCE & DnS Ultralight (DUL). •Non Ontological Resource reused: N/A
M3O
•It aims at integrating data resources related to media, especiallythose used on the Web. W3C initiative, OWL.•Scope covered: Multimedia, audio/music, video.•Ontological Resource reused: N/A. •Non Ontological Resource reused: SKOS
Media Onto
•MPEG-7_Hunter, MPEG-7x , MPEG-7_Tsinakari, MPEG-7_Rhizomik . [2001- 2006]•SWintO (mobile access, 2007): Multimedia, image, video. (RDFS)•Ontological Resource reused: DOLCE, SUMO. •Non Ontological Resource reused: MPEG 7
MPEG7 transformations
+ SWintO
Analyzing and Ranking Multimedia Ontologies for their Reuse
Analyzing and Ranking Multimedia Ontologies for their Reuse
Ontologies for describing Shapes and Image
14
•Ontology for describing metadata for digital images, [Raphael Troncy + Ughent University, 07], OWL Full.•Scope covered: image.•Ontological Resource reused: N/A. •Non Ontological Resource reused: IIA
DIG 35
•It defines concepts including image, video, video frame, region, as well as relations such as depicts, regionOf, etc.,[Halaschek-Wiener et. al., 2005], OWL Full.•Scope covered: video, image•Non Ontological Resource reused: N/A
MIRO
•It combines high-level domain concepts and low-level multimedia descriptions, enabling for new media content analysis . [Kosmas Petridis et al., 2007] aceMedia Project ,OWL.•Scope covered: Multimedia, video.•Ontological Resource reused: DOLCE, MPEG-7(MDS)
MSO
•Metadata for any kind of shape •Creation and processing digital shapes. AM@SHAPE , 2005•Image, visual. (OWL Full)•Ontological Resource reused: N/A.
CSO, SAPO
Analyzing and Ranking Multimedia Ontologies for their Reuse
Analyzing and Ranking Multimedia Ontologies for their Reuse
Ontologies for describing Visual Resource Object
15
•VRA is an Asociation maintining collections of slides, images and works of art. •Two versions of the ontology: SIMILE project (2003,RDFS ) and Assem (2005,OWL)•Scope covered: image, visual•Ontological Resource reused: N/A. •Non Ontological Resource reused: VRA Element Set
Vra Core 3
•Deals with semantic MM content, analysis and reasoning.•Developed within aceMedia Project, 2005•Scope covered: video, image.•Ontological Resource reused: DOLCE extension. •Non Ontological Resource reused: MPEG-7
VDOVDO
Analyzing and Ranking Multimedia Ontologies for their Reuse
Analyzing and Ranking Multimedia Ontologies for their Reuse
Ontologies for describing Audio and Music
16
•Vocabulary for linking music-related information (production process, temporal aspect and events in music)•[Frederick Giasson, Yves Raimonf, 2010] in RFS•Scope covered: audio•Ontological Resource reused: Foaf, Time, Event, TimeLine •Non Ontological Resource reused: ABC Data Model.
Music Onto
•It describes classical music and performance.•Difference between musical works (e.g. Ballet) from performance (Ballet_Event), or works (Choral_Music)•[Kanzaki, 2005], OWL DL.•Scope covered: audio•Non Ontological Resource reused: N/A
Kanzaki’s Music Vocab
•It describes artists, music titles and some descriptors from the audio (tonality, rhythm, tempo )•[Oscar Celma, 2006] ,OWL DL•Scope covered: audio•Ontological Resource reused: FOAF•Non Ontological Resource reused: RDF Site Summary (RSS)
Recommendation
OntologyMusic
Analyzing and Ranking Multimedia Ontologies for their Reuse
Analyzing and Ranking Multimedia Ontologies for their Reuse
Application Ontologies
17
•Aims at cross-link of media campaigns over media TV, press and Internet•[MediaCampaign, 2006] in RFS•Scope covered: audiovisual•Ontological Resource reused: PROTON•Non Ontological Resource reused: NewsML, News Codes
MEPCO
•It describes athletics events (e.g. jumping, running, etc.) held in European cities.•[Boemie, 2008], OWL DL.•Scope covered: Multimedia, visual•Ontological Resource reused: GIO•Non Ontological Resource reused: TeleAtlas DB, MPEG-7, IAAF
AEO
•It aims at providing virtual representations of humans•[AM@SHAPE, 2007] ,OWL Full•Scope covered: image, visual•Ontological Resource reused: CSO•Non Ontological Resource reused: RDF Site Summary (RSS)
VHO
Analyzing and Ranking Multimedia Ontologies for their Reuse
Analyzing and Ranking Multimedia Ontologies for their Reuse
Conclusion SoA
• None of existing ontology integrate both low level descriptions (e.g., color, textures, fragments, etc.) and high level descriptions (voice, videoclip, slides presentation, domain content, etc.) of MM resources in all its five aspects (audio, video, image, visual, audiovisual, multimedia)
• None of the existing ontology describes MM resources in different domains and in different natural languages.
18Analyzing and Ranking Multimedia Ontologies for their Reuse
19Analyzing and Ranking Multimedia Ontologies for their Reuse
Outline
• Introduction
• State of the Art on MultiMedia Ontologies
• Searching MM Ontologies
• Assessing MM Ontologies
• Selecting MM Ontologies
• Conclusions
20Analyzing and Ranking Multimedia Ontologies for their Reuse
Semantic Web Engines (SWEs)
Semantic Web Engines are applications for finding ontologies where queries are usually written as natural language keywords and results are ranked.
RDF-based search engines
Ontology-based search engines
Hybrid-based search engine
21Analyzing and Ranking Multimedia Ontologies for their Reuse
Selection of the most appropriate SWE
The total number of documents retrieved (T) for a specific keyword search.
The number of OWL documents per each 10 documents (OWL).
A valoration of the retrieval results using the symbols (+) and (-) of the result. We set to (+) if there are more than 2 OWL files per page, and (-) otherwise.
Terms used: Image, Multimedia, Audio, Music Style, Format
Set of criteria
SwoogleSwoogle
22
Searching ontologies based on requirements
ANALYZING AND RANKING MULTIMEDIA ONTOLOGIES FOR THEIR REUSE
Functional requirements
ORSD
Non Functional requirements
23Analyzing and Ranking Multimedia Ontologies for their Reuse
Tasks for searching MM ontologies (I)
Terms translated into English
Terms extracted from the ORSD
24Analyzing and Ranking Multimedia Ontologies for their Reuse
Tasks for searching MM ontologies (II)
25Analyzing and Ranking Multimedia Ontologies for their Reuse
Tasks for searching MM ontologies (III)
But there are missing ontologies from SoA!!
25 ontologies retrieved with Swoogle
26Analyzing and Ranking Multimedia Ontologies for their Reuse
Tables of candidate MM ontologies: Unification process
List of 40 ontologies:
SWE + SoA
27Analyzing and Ranking Multimedia Ontologies for their Reuse
Outline
• Introduction
• State of the Art on MultiMedia Ontologies
• Searching MM Ontologies
• Assessing MM Ontologies
• Selecting MM Ontologies
• Conclusions
28Analyzing and Ranking Multimedia Ontologies for their Reuse
Analysis based on requirements (I)1-The competency questions (CQs) and one ontology selected from the searching activity. The result is a set of CQs identifiers that cover the given ontology.
3-Open the ontology to analyze in the Neon Toolkit. Open also the document with the list of CQs.
29Analyzing and Ranking Multimedia Ontologies for their Reuse
Analysis based on requirements (II)
4- For each CQs, detect the relevant categories and create a list of "Relevant Categories" (RelevCat). Example: "What are Audio Format", with the answer: "AVI, MP3"; RelevCat={Format, Audio, AVI, MP3}.
5- The matching task consists of finding for each term of the relevant categories, its presence in the ontology as a class or an individual. Update (CQ identifier)
30Analyzing and Ranking Multimedia Ontologies for their Reuse
Assessment table/ ”useful” ontologies
Heuristic
IF (SimilarScope) OR (Similar Purpose) OR (Functional RequirementsCovered) = No
Then
NotUseful (CandidateOntology)EliminateFromSetCandidate(CandidateOntology)
Some wrong situationsSome wrong situations 26 “useful” ontologies:12: SoA
14: SWE
26 “useful” ontologies:12: SoA
14: SWE
[Suárez-Figueroa, 2010]
31Analyzing and Ranking Multimedia Ontologies for their Reuse
Outline
• Introduction
• State of the Art on MultiMedia Ontologies
• Searching MM Ontologies
• Assessing MM Ontologies
• Selecting MM Ontologies
• Conclusions
32
Criteria for selecting MM ontologies
Analyzing and Ranking Multimedia Ontologies for their Reuse
[Suárez-Figueroa, 2010]
M.C. Suárez-Figueroa. PhD Thesis: NeOn Methodology for Building Ontology Networks: Specification, Scheduling and Reuse. Spain. Universidad Politécnica de Madrid. Junio 2010.
33Analyzing and Ranking Multimedia Ontologies for their Reuse
Determining the most appropriate MM ontologies. Considerations
1. Easy accessibility of the ontologies
2. Most of the ontologies were developed within a project or institutional initiatives (e.g: Boemie), highest scores in the Quality of the documentation, availability of external knowledge, and code clarity. The rest are made by academic researchers.
3. Some ontologies were developed or transformed by one author reputation and purpose reliability lower than others ontologies.
4. In ”practical support”, very relevant others publications referencing the ontology or the use of the same ontology in a large project (e.g.: COMM, SAPO, Boemie VDO)
5. Difficult to know if the ontologies were tested and/or evaluated after their implementation
34
Determining the most appropriate MM ontologies (I)
Analyzing and Ranking Multimedia Ontologies for their Reuse
35
Determining the most appropriate MM ontologies (II)
Analyzing and Ranking Multimedia Ontologies for their Reuse
)()( ,
jj
j
jTi Weight
WeightValueScore
ji
)(,)(, iii ScoreScoreScore
Value = Unknown ValueT = 0Value = Low ValueT = 1Value = Medium ValueT = 2Value = High ValueT = 3
Formulae to rank ontologies
[Suárez-Figueroa, 2010]
36
Integrating the MM ontologies reused. General vision
Analyzing and Ranking Multimedia Ontologies for their Reuse
Music Ontology: 1 CQ covered
Media Ontology: 4 CQs coveredMedia Ontology: 4 CQs covered
COMM: 5 CQs coveredCOMM: 5 CQs covered
Boemie VDO: 4 CQs coveredBoemie VDO: 4 CQs covered
They cover 70% of the CQs!!
They cover 70% of the CQs!!
37
Integrating the MM ontologies reused. Overview of the M3 Ontology
Analyzing and Ranking Multimedia Ontologies for their Reuse
38Analyzing and Ranking Multimedia Ontologies for their Reuse
Outline
• Introduction
• State of the Art on MultiMedia Ontologies
• Searching MM Ontologies
• Assessing MM Ontologies
• Selecting MM Ontologies
• Conclusions
• We used the NeOn Methodology [Suárez-Figueroa, 2010] to perform a systematic analysis of all the candidate ontologies.
• Methodological guidelines for reusing domain ontologies.
• More specifically, we have been focused on:
• (1) searching for ontological resources in repositories and registries 40 ontologies found.
• (2) assessing the ontological resources in order to find out if such resources satisfy the developers needs 23 ontologies obtained.
• (3) comparing the ontological resources on the basis of a set of criteria and selecting the most appropriate ones based on the requirements: improving and extending them with specific rules to analyze the ontologies Ontologies ranked.
• (4) integrating the ontological resources: selection of 4 suitable ontologies to be reused in the M3 Ontology.
39
Conclusions
What we have done in this master thesis
Analyzing and Ranking Multimedia Ontologies for their Reuse
M.C. Suárez-Figueroa. PhD Thesis: NeOn Methodology for Building Ontology Networks: Specification, Scheduling and Reuse. Spain. Universidad Politécnica de Madrid. June 2010.
• Searching activity:• An overview of MM ontologies in the literature.
• How to select an appropriate SWE to retrieve relevant ontologies in the MM domain.
• Workflow to search relevant ontologies based on set of terms extracted from the CQs.
• Assessing activity:• A comparative framework for MM ontologies.
• Workflow to check if an ontology fits the requirements.
• Selecting activity:• Adaptation of the criteria for selection proposed by [Suárez-Figueroa, 2010]
to the MM domain.
• Inclusion of a new criteria.
• Integration and implementation of the M3 Ontology.
40
Conclusions (II)
Main contributions
Analyzing and Ranking Multimedia Ontologies for their Reuse
Many of the processes described in the ontology reuse activities are described in natural language need to be formalized and automatized.
Searching activity is not exclusive to the used Semantic Search Engines, and must be extended to articles, project web pages, and W3C groups related to the domain.
Semantic Web Engines do not clearly distinguish in the results from keywords queries, RDF data coming from blogs and DBPedia resources to ontologies documents implemented in OWL.
Some criteria proposed in [Suárez-Figueroa, 2010] concerning ranking ontologies need to be adapted to the domain of the ontology being developed.
There is lack of multilingual ontologies in multimedia domain.
41
Conclusions (III)
Lessons learned
M.C. Suárez-Figueroa. PhD tesis: NeOn Methodology for Building Ontology Networks: Specification, Scheduling and Reuse. Spain. Universidad Politécnica de Madrid. Junio 2010.
Analyzing and Ranking Multimedia Ontologies for their Reuse
42
Future Work (I)
How to choose the right SWE that gives better results? What could be the criteria that guide deciding which SWE to use in function of the domain?
CQs Enhancing Ontology Search Tasks:
Many SWE presents their results mixing documents from blogs with ontologies. An API can help the developer to extract efficiently disseminated ontologies in the whole documents retrieved by a SWE.
How to select the right ontology from the results retrieved by search engines
With the continuously growing of the DBPedia resources, analyze how to populate ontologies and their reliability with respect to the one to be built.
Semi-automatic ontology population:
Analyzing and Ranking Multimedia Ontologies for their Reuse
43
End
Thanks!
Analyzing and Ranking Multimedia Ontologies for their Reuse
Date: 11/04/23Speaker: Ghislain Auguste Atemezing
ANALYZING AND RANKING MULTIMEDIA ONTOLOGIES FOR THEIR REUSE
Master ThesisMáster de investigación en inteligencia artificial
Author: Ghislain Auguste Atemezing
Supervisor: Dr. María del Carmen Suárez de Figueroa Baonza
top related