dog i : an annotation system for images of dog breeds antonis dimas pyrros koletsis euripides...
Post on 24-Dec-2015
214 Views
Preview:
TRANSCRIPT
DOGI: an Annotation System for Images of Dog Breeds
Antonis Dimas
Pyrros Koletsis
Euripides Petrakis
Intelligent Systems Laboratory
Technical University of Crete (TUC)
Chania, Crete, Greece
Image Annotation
The task of assigning a name or description to an unknown image
Manual: good quality, but slow, subjective Automatic: classification problem, relies on
associating image analysis features with high level concepts Difficult to handle all image types Semantic gap: map features to classes
04/19/23 ICIAR 2012, Aveiro, Portugal 2
DOGI: http://www.intelligence.tuc.gr/prototypes.php
An automatic image annotation system for images of dog breeds 40 classes (dog breeds) Descriptions: information in an ontology Class names, properties, features, textual
descriptions (from WordNet, Wikipedia) Annotations in MPEG7
04/19/23 ICIAR 2012, Aveiro, Portugal 3
DOGI: System
04/19/23 ICIAR 2012, Aveiro, Portugal 4
Graphical User Interface (GUI)
Feature Extraction
Ontology Mapping
Image Annotation
DOGI
Ontology
Load ImageSelect ROIAnnotation Method
MPEG7 features Color + Texture featuresImages: 12-dim vectors
40 classes9 instances/classClass hierarchy Class properties
Image RetrievalSelect Annotation MathodStore Annotation in Exif header
Select ROI
04/19/23 ICIAR 2012, Aveiro, Portugal 5
Image Content Analysis
Images of dog breeds are mainly characterized by the spatial distribution of color intensities
A 12-dimension feature vector of Color, Texture, Hybrid feature from LIRE Library
Features are normalized in [0,1] Not all features are equally important
04/19/23 ICIAR 2012, Aveiro, Portugal 6
Ontology
40 classes of dog breed organized in IS_A hierarchy E.g., Dog Working Group Saint Bernard
Three separate hierarchies for text, features and visual descriptions
9 instances per class: raw images + a 12-dim feature vector for each image in class
04/19/23 ICIAR 2012, Aveiro, Portugal 7
DOGI Ontology
04/19/23 ICIAR 2012, Aveiro, Portugal 8
Image Annotation
The unknown image Q is compared with each one of the 360 images in the ontology
D(Q,I) = Σi widi(Q,I)
Results are ranked by similarity with Q Weights wi are computed by decision trees
Training set of 3,474 image pairs
04/19/23 ICIAR 2012, Aveiro, Portugal 9
i feature of nodes
1
i
)(1maxdepth
)uredepth(feat - 1 maxdepth
nodesall
j j
inodedepth
w
Annotation Method
Best Match: Select class of most similar instance
Max Occurrence: Select class with more instances in the first 20 answers
Average Retrieval Rank: Select class with instances ranked higher in the first 20 answers
Max Similarity: Select class whose instancing sum-up to max similar score
04/19/23 ICIAR 2012, Aveiro, Portugal 10
Example Image
04/19/23 ICIAR 2012, Aveiro, Portugal 11
Annotation Result
04/19/23 ICIAR 2012, Aveiro, Portugal 12
EXIF Metadata
Descriptive information embedded inside an image
The metadata captured by your camera is called EXIF data ..
DOGI stores annotation info with the pictures in the EXIF
Can be useful for image archiving and later retrievals
04/19/23 ICIAR 2012, Aveiro, Portugal 13
Annotation in MPEG7
04/19/23 ICIAR 2012, Aveiro, Portugal 14
Evaluation
Average annotation accuracy over 40 queries
04/19/23 ICIAR 2012, Aveiro, Portugal 15
Annotation result
Max Similarity
AVR Max Occurrence
Best Match
Ranked 1st 72.5% 62.5% 65% 50%
Ranked 2nd 17.5% 22.5% 15% 10%
Ranked 3rd 5% 10% 10% 10%
Overall 95% 92,5% 90% 90%
Conclusions-Future Work
DOGΙ : An automatic annotation system for dog breeds with good performance
Useful as a tool for many application Annotation accuracy improves for less
categories Experimenting with more and animal species
images categories More elaborate image classification methods
04/19/23 ICIAR 2012, Aveiro, Portugal 16
THANK YOU !!
04/19/23 ICIAR 2012, Aveiro, Portugal 17
top related