mediaeval 2012 placing task overview
TRANSCRIPT
![Page 1: MediaEval 2012 Placing Task Overview](https://reader030.vdocuments.mx/reader030/viewer/2022032503/55bfce75bb61eb7e458b4808/html5/thumbnails/1.jpg)
Placing TaskOrganisers: Adam Rae (Yahoo! Research)
Pascal Kelm (Technische Universität Berlin)
Smile!
?
!
![Page 2: MediaEval 2012 Placing Task Overview](https://reader030.vdocuments.mx/reader030/viewer/2022032503/55bfce75bb61eb7e458b4808/html5/thumbnails/2.jpg)
Task Description
• Given a video, how accurately can it be placed on a map and be given latitude and longitude coordinates?
METADATA
![Page 3: MediaEval 2012 Placing Task Overview](https://reader030.vdocuments.mx/reader030/viewer/2022032503/55bfce75bb61eb7e458b4808/html5/thumbnails/3.jpg)
Task Overview
• Automatic location annotation of online videos
• 7 teams submitted results (17% up)– 5 veterans– 2 new participants
• First year for code sharing– GitHub (currently)
![Page 4: MediaEval 2012 Placing Task Overview](https://reader030.vdocuments.mx/reader030/viewer/2022032503/55bfce75bb61eb7e458b4808/html5/thumbnails/4.jpg)
Data
• Provided– Textual metadata: tags, titles, descriptions– Visual: 9 visual features extracted for key frames
every 4 seconds– Additional media: images with textual and
visual feature data• Available (external)– Up to the participant, but controlled according to
run submission
![Page 5: MediaEval 2012 Placing Task Overview](https://reader030.vdocuments.mx/reader030/viewer/2022032503/55bfce75bb61eb7e458b4808/html5/thumbnails/5.jpg)
Data
• Training– 15,563 videos (combination of last year’s training
and test data)– 3,185,258 additional Flickr images
• Test– 4,182 videos
2012 Training
2011 Training
2010 Training
2010 Test2011 Test
2012 Test
![Page 6: MediaEval 2012 Placing Task Overview](https://reader030.vdocuments.mx/reader030/viewer/2022032503/55bfce75bb61eb7e458b4808/html5/thumbnails/6.jpg)
Evaluation
• Take the latitude + longitude suggested by participants for each video
• Compute Haversine distance between that and the ‘true’ location
• We group results into buckets of increasing radii, e.g. 1km, 10km, 20km, etc.
![Page 7: MediaEval 2012 Placing Task Overview](https://reader030.vdocuments.mx/reader030/viewer/2022032503/55bfce75bb61eb7e458b4808/html5/thumbnails/7.jpg)
Overall Best Results
CEALIST
IRISA
UNICAMP
GENT
TUB
ICSI
TUD
0% 5% 10% 15% 20% 25% 30%
Percentage of correct locations @ 1km
Organiser-connected team
![Page 8: MediaEval 2012 Placing Task Overview](https://reader030.vdocuments.mx/reader030/viewer/2022032503/55bfce75bb61eb7e458b4808/html5/thumbnails/8.jpg)
1 10 100 1000 10000 1000000
500
1000
1500
2000
2500
3000
3500
4000
4500
Only Restriction: No new material, gazetteer permitted
ICSI TUD UG-CUUNICAMP CEA_LIST London Baseline
Distance from Ground Truth
Corr
ect T
est V
ideo
s
![Page 9: MediaEval 2012 Placing Task Overview](https://reader030.vdocuments.mx/reader030/viewer/2022032503/55bfce75bb61eb7e458b4808/html5/thumbnails/9.jpg)
1 10 100 1000 10000 1000000
500
1000
1500
2000
2500
3000
3500
4000
4500
Restriction: Visual Only
CEA_LIST ICSI IRISA UG-CU UNICAMP TUB
Distance from Ground Truth
Corr
ect T
est V
ideo
s
![Page 10: MediaEval 2012 Placing Task Overview](https://reader030.vdocuments.mx/reader030/viewer/2022032503/55bfce75bb61eb7e458b4808/html5/thumbnails/10.jpg)
Detected trends and activity of note
• What classes of approaches were taken (has this change since last year?)– Textual, visual– Graph modelling– User modelling– …combinations of above
• Challenging Assumptions– Spatial locality visual stability?
• Absolute performance lower than last year – but…– Different data set– Less textual metadata in general
![Page 11: MediaEval 2012 Placing Task Overview](https://reader030.vdocuments.mx/reader030/viewer/2022032503/55bfce75bb61eb7e458b4808/html5/thumbnails/11.jpg)
Future of the task
• Still room for improvement• Still a valuable task?• Standard of science improving
• Need new organisers! Talk to Pascal and me