Download - Reliving on demand a total viewer experience
![Page 1: Reliving on demand a total viewer experience](https://reader033.vdocuments.mx/reader033/viewer/2022052623/559e1cb01a28abd75b8b479b/html5/thumbnails/1.jpg)
1
RELIVING ON DEMAND: A TOTAL VIEWER
EXPERIENCE
Vivek K. Singh1*, Jiebo Luo2, Dhiraj Joshi2, Phoury Lei2, Madirakshi Das2, Peter Stubler2
ACM International Conference on Multimedia – ACMM 2011
1 University of California, Irvine, 2 Kodak Research Laboratories, Rochester, NY,
* Work was done when the author was interning at Kodak Research Laboratories, Eastman Kodak Company, Rochester, NY, USA.
![Page 2: Reliving on demand a total viewer experience](https://reader033.vdocuments.mx/reader033/viewer/2022052623/559e1cb01a28abd75b8b479b/html5/thumbnails/2.jpg)
Why do people take pictures?
1. Digital re-living
2. Sharing it with
family and friends
![Page 3: Reliving on demand a total viewer experience](https://reader033.vdocuments.mx/reader033/viewer/2022052623/559e1cb01a28abd75b8b479b/html5/thumbnails/3.jpg)
What’s available today?
• Commercial Slideshows (Picasa, iPhoto, ACDsee):
• Focus on visual appearance only.
• Don’t understand/utilize semantics (except “FaceMovie”)
• Research efforts: Semantic analysis
• No interaction
• Interaction on demand
• Allow different users to dynamically re-direct the flow of
media reliving experience
![Page 4: Reliving on demand a total viewer experience](https://reader033.vdocuments.mx/reader033/viewer/2022052623/559e1cb01a28abd75b8b479b/html5/thumbnails/4.jpg)
Platforms
Desktop
Digital frame
HDTV
Kodak Gallery
Mobile
Kiosk
![Page 5: Reliving on demand a total viewer experience](https://reader033.vdocuments.mx/reader033/viewer/2022052623/559e1cb01a28abd75b8b479b/html5/thumbnails/5.jpg)
Preview
• Re-living of events in user’s life, based on WHO,
WHERE, and WHEN .
![Page 6: Reliving on demand a total viewer experience](https://reader033.vdocuments.mx/reader033/viewer/2022052623/559e1cb01a28abd75b8b479b/html5/thumbnails/6.jpg)
Outline
• Preview
• Design principles
• System design
• Under the hood (sneak peek)
• Evaluations
![Page 7: Reliving on demand a total viewer experience](https://reader033.vdocuments.mx/reader033/viewer/2022052623/559e1cb01a28abd75b8b479b/html5/thumbnails/7.jpg)
Design principles
1. User controllable:
• Responsive to user demand (overcoming intent gap)
2. Semantically drivable:
• Events as organizing units
• Who, when, where; what
3. Aesthetically pleasing:
• Dynamic presentation
• Multimodal (songs, images, videos)
![Page 8: Reliving on demand a total viewer experience](https://reader033.vdocuments.mx/reader033/viewer/2022052623/559e1cb01a28abd75b8b479b/html5/thumbnails/8.jpg)
Retrieval vs. Browsing vs. Reliving
• Media by itself is uninteresting unless it performs a
function (e.g. reliving, sharing) for the human user
• Retrieval
• Fetching data. Strong intent (e.g. search)
• Browsing
• Piecemeal reliving. Weak intent (e.g. youtube)
• Reliving
• Valuable middle ground.
• Semantically re-direct the flow if desired.
![Page 9: Reliving on demand a total viewer experience](https://reader033.vdocuments.mx/reader033/viewer/2022052623/559e1cb01a28abd75b8b479b/html5/thumbnails/9.jpg)
System overview
![Page 10: Reliving on demand a total viewer experience](https://reader033.vdocuments.mx/reader033/viewer/2022052623/559e1cb01a28abd75b8b479b/html5/thumbnails/10.jpg)
System overview: Approach
![Page 11: Reliving on demand a total viewer experience](https://reader033.vdocuments.mx/reader033/viewer/2022052623/559e1cb01a28abd75b8b479b/html5/thumbnails/11.jpg)
Media data structure
TypeURL
Aesthetic IVI
dateTimesubjects
location
Height, width
Media
properties
Aesthetic
properties
Semantic
properties
Score Suitability
properties
![Page 12: Reliving on demand a total viewer experience](https://reader033.vdocuments.mx/reader033/viewer/2022052623/559e1cb01a28abd75b8b479b/html5/thumbnails/12.jpg)
Pre-processingMedia
Collection
Metadata
Repository
Date and Time
Extraction
Location Information
Extraction
Event Clustering
Aesthetics Value
Extraction
Face Detection
Face Clustering
Geographic
Clustering
Face Labeling
![Page 13: Reliving on demand a total viewer experience](https://reader033.vdocuments.mx/reader033/viewer/2022052623/559e1cb01a28abd75b8b479b/html5/thumbnails/13.jpg)
Reordering of event list
• Basic idea
• Time
• People
• Location
![Page 14: Reliving on demand a total viewer experience](https://reader033.vdocuments.mx/reader033/viewer/2022052623/559e1cb01a28abd75b8b479b/html5/thumbnails/14.jpg)
Choosing layout
• Default:
i = 2 3 4 5
![Page 15: Reliving on demand a total viewer experience](https://reader033.vdocuments.mx/reader033/viewer/2022052623/559e1cb01a28abd75b8b479b/html5/thumbnails/15.jpg)
Choose transitions
• If (criteria=time || criteria=loc)
• Slide In/Out
• If (criteria=personi)
• Face2Face transition
Transform(θ1, trans.X
1,
trans.Y 1, scale 1)
Transform(θ2, trans.X
2,
trans.Y2, scale 2)
![Page 16: Reliving on demand a total viewer experience](https://reader033.vdocuments.mx/reader033/viewer/2022052623/559e1cb01a28abd75b8b479b/html5/thumbnails/16.jpg)
Choose song
• If (criteria=time)
• Select seasonal songs (easily extensible to finer grain)
• If (criteria=loc)
• Select regional songs
• If (criteria=personi)
• Select age-based songs (easily extensible to gender)
• Taken from a library of available songs
![Page 17: Reliving on demand a total viewer experience](https://reader033.vdocuments.mx/reader033/viewer/2022052623/559e1cb01a28abd75b8b479b/html5/thumbnails/17.jpg)
Show images
• In time order
• Higher score => more display time
• Auto-zoom-crop
• Find center to focus on
• Match the aspect ratio required
• Multiple Holes in transitions
• Token passing amongst holes
• Representative image as background
![Page 18: Reliving on demand a total viewer experience](https://reader033.vdocuments.mx/reader033/viewer/2022052623/559e1cb01a28abd75b8b479b/html5/thumbnails/18.jpg)
Logging user sessions<Interaction>
<Click>
<GlobalEventID>urn:guid:f1337996-3c28-4345-b4fb-c4f1b788fc05</GlobalEventID>
<SortedEventID>0</SortedEventID
<TimeStamp>10:17:47 AM</TimeStamp>
<Criteria_type>gps</Criteria_type>
<Criteria_value>61.2175937710438 , -149.898739309764</Criteria_value>
<HotSpotClick>False</HotSpotClick>
</Click>
<Snapshot>
<Locations>
<loc>-149.898739309764,61.2175937710438</loc>
<loc>-73.508556462585,40.5956603174603</loc>
<loc>102.757525301205,25.1018832329317</loc>
<loc>104.195397,35.86166</loc>
<loc>6.09306585111111,52.7236709366667</loc>
</Locations>
<People>
<peo>Jiebo</peo>
<peo>Joyce</peo>
<peo>Xinping</peo>
<peo></peo>
<peo></peo>
</People>
<SortedEvents>
<eve>urn:guid:f1337996-3c28-4345-b4fb-c4f1b788fc05</eve>
<eve>urn:guid:f1337996-3c28-4345-b4fb-c4f1b788fc05</eve>
<eve>urn:guid:f1337996-3c28-4345-b4fb-c4f1b788fc05</eve>
<eve>urn:guid:f1337996-3c28-4345-b4fb-c4f1b788fc05</eve>
<eve>urn:guid:f1337996-3c28-4345-b4fb-c4f1b788fc05</eve>
<eve></eve>
</SortedEvents>
<PicsShown>
<pic>c:\data\jiebo\cvpr2008\103_5972.jpg</pic>
<pic>c:\data\jiebo\cvpr2008\103_5973.jpg</pic>
<pic>c:\data\jiebo\lijiang-shangrila-day2\108_0043.jpg</pic>
<pic>c:\data\jiebo\lijiang-shangrila-day2\108_0044.jpg</pic>
</PicsShown>
</Snapshot>
</Interaction>
![Page 19: Reliving on demand a total viewer experience](https://reader033.vdocuments.mx/reader033/viewer/2022052623/559e1cb01a28abd75b8b479b/html5/thumbnails/19.jpg)
Evaluations
• Experiments with 11 families
• 35 user interaction sessions logged
• Roles
• 1st person (owner)
• 2nd person (immediate family)
• 3rd person (friends, cousins )
Age of contributing photographers 23 to 56
No. of images/ videos in the collection 2,091 to 10,522
No. of calendar years in time span 3 to 10
No. of tagged people in the collection 26 to 137
No. of places in the collection 19 to 45
![Page 20: Reliving on demand a total viewer experience](https://reader033.vdocuments.mx/reader033/viewer/2022052623/559e1cb01a28abd75b8b479b/html5/thumbnails/20.jpg)
Experiment 1: Comparison with commercially available
options
![Page 21: Reliving on demand a total viewer experience](https://reader033.vdocuments.mx/reader033/viewer/2022052623/559e1cb01a28abd75b8b479b/html5/thumbnails/21.jpg)
6.2 Experiment 2: Use of different features across
different user demographics
Females 1.14 1.49 1.13 1.01
Males 1.41 1.25 2.08 1.43
Both 1.30 1.27 1.28 1.35
All 1st party 2nd party 3rd party
Active Vs Passive?
Clicks per axis Stickiness :Time spent after clicks
![Page 22: Reliving on demand a total viewer experience](https://reader033.vdocuments.mx/reader033/viewer/2022052623/559e1cb01a28abd75b8b479b/html5/thumbnails/22.jpg)
Future work
• Choosing songs more generically/smartly
• Choosing optimal spatio-temporal placement of
images in the slide show
• Choosing layout
• Choosing transition time?
• Supporting multiple axes simultaneously
• Previews