Selecting User-Generated Videos for Augmented Reality Applications
Hien To, Hyerim Park, Seon Ho Kim, and Cyrus ShahabiIntegrated Media Systems Center
University of Southern California, Los Angeles, CA
BigData in Social Media
IMSC Retreat 2016
IntroductionMobile cameras are everywhere!Any way to use mobile videos as AR content?
Motivation• Popularity of augmented-reality (AR)
AR browsers, e.g., Layar, Wikitude, and Junaio• Need content source with rich context information
Scarcity of AR contentLack of user contexts causes imprecise registration
Challenges• Hard to search for interesting videos for AR content
Proposed ApproachSpatial metadata
Identify interesting videos• Sequence of FOVs that follow a particular camera
shooting pattern
Modeling significant video segments
References• Incorporating Geo-Tagged Mobile Videos Into Context-Aware
Augmented Reality Applications, IEEE BigMM 2016.• GeoUGV: User-Generated Mobile Video Dataset with Fine
Granularity Spatial Metadata, MMSys 2016.
Results1. Significant Video Segments
2. Detecting Hot Spots
ConclusionProposed models and algorithms to identify
interesting video segments and hotspotsThe algorithms are fast and able to identify
interesting scenesWill consider user mobility and ranks of video
segments
�⃗��⃗�
pR𝜃
Field Of View (FOV)p: camera locationd: camera direction: viewable angleR: viewable distancet: timestamp
Single MultipleSingle
Zoom Pan
MultipleTrack Arch
PositionDirection
1d2d
3d
4d5d
1d2d 3d 4d
5d
1d
2d3d
4d
5d1d
2d3d
4d 5d
34 5
0
¿𝑡𝑚𝑖𝑛 ,𝑑𝑚𝑎𝑥>¿
¿ 𝑡𝑚𝑖𝑛 ,𝑑𝑚𝑎𝑥 ,𝑟𝑚𝑎𝑥>¿ ¿ 𝑡𝑚𝑖𝑛 ,𝑑𝑚𝑖𝑛 ,𝑟𝑚𝑎𝑥>¿
¿ 𝑡𝑚𝑖𝑛 ,𝑑𝑚𝑎𝑥>¿
Number of scenes found
Track Arch Zoom Pan0
500
1000
1500
2000
Num
ber o
f sce
nes
Searching time (one video)
Track Arch Zoom Pan0
5
10
15
20
25
30
Run
time
(ms)
Tracking is the most popular way of capturing mobile videos Overall search time is fast
Hotspots covered by many video frames
100
90 80 70 60 50 40 30 20 100
200400600800
1000
Minimum FOVs per hotspot
Tota
l num
ber o
f hot
sp
ots
Real Dataset
Statistics ValueNumber of videos with geo-metadata 2,397Average length per video with content (sec) 72.14Number of users 289Number of FOVs 208,978
Zoom Pan
Track Arch
