what's in a place? adventures with location-aware media
DESCRIPTION
Can we automatically create an “attraction map” of the world from Flickr geotagged images and their associated tags? We performed an analysis of Flickr data and developed a visualization technique called Tag Maps to do exactly that. Using the analysis and the Tag Maps visualization, we created an exploration tool called World Explorer that allows one to, well, explore the world like never before. The idea behind the data analysis is simple: by taking a photo, photographers essentially express their interest in a particular place, and implicitly “vote” in favor of that location. This gives us a set of highly representative tags associated with each map location. The World Explorer visualization is facilitated by placing these representative tags on a map (“a Tag Map”). We augment the Tag Map with photos that represent each tag at its specific location. Together, World Explorer effectively provides a sense of the important concepts and attractions embodied in each map area and zoom level, and allows users – tourists planning a trip, virtual world-discoverers or just some bored individuals – to explore the world via photos. I’ll also give a brief demo and overview of Zurfer, a novel mobile phone context-aware software prototype that enables access to images on the go. It utilizes the channel metaphor to give users contextual access to media of interest according to key dimensions: spatial, social, and topical. Zurfer attempts to be playful and simple to use, yet provide powerful and comprehensive media access. A temporally-driven sorting scheme for media items allows quick and easy access to items of interest in any dimension. For novice users, and more complicated tasks, we extend the application incorporating keyword search to deliver the long tail of media and images. http://www.rahulnair.net/blog/2007/06/07/talking-at-ricoh-what%e2%80%99s-in-a-place/TRANSCRIPT
![Page 1: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/1.jpg)
1yahooresearchberkeley.comRahul Nair
What’s in a place?Adventures with Location-Aware Media
Rahul Nair
Yahoo! Research Berkeley
![Page 2: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/2.jpg)
2yahooresearchberkeley.comRahul Nair
Flickr “geotagged”
20+ million images
Can we do better?
![Page 3: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/3.jpg)
3yahooresearchberkeley.comRahul Nair
Talk Outline
• Extracting information from geo-tagged photos
• World Explorer: Visualization
• ZoneTag: Creation
• Zurfer: Consumption
![Page 4: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/4.jpg)
4yahooresearchberkeley.comRahul Nair
Attraction Map of Paris
Stanley Milgram, 1976. Psychological Maps of Paris
![Page 5: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/5.jpg)
5yahooresearchberkeley.comRahul Nair
Attraction Map of Paris
Y!RB, 2007.
![Page 6: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/6.jpg)
6yahooresearchberkeley.comRahul Nair
Location-driven Modeling
• Derive meaningful data about map regions
• E.g., representative tags, photos
![Page 7: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/7.jpg)
7yahooresearchberkeley.comRahul Nair
Data Description
![Page 8: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/8.jpg)
8yahooresearchberkeley.comRahul Nair
Issues
• Sparse data set
• Photographer bias
– In location
– In tags
• Incorrect data
![Page 9: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/9.jpg)
9yahooresearchberkeley.comRahul Nair
Heuristics
• Number of photographs denotes the “importance” of a location
• Users will use a common subset of tags to describe objects/locations
• Concentrated tag usage indicates descriptiveness
![Page 10: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/10.jpg)
10yahooresearchberkeley.comRahul Nair
Algorithm
• Clustering: k-Means, get set of k clusters
• “Document” C is bag of all tags in cluster
• For each tag in C calculate:
– TF = |P(C,t)|
– IDF = |P(R)| / |P(R, t)|
– UF = |U(C,t)|/|U(C)|
![Page 11: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/11.jpg)
11yahooresearchberkeley.comRahul Nair
Scoring
• Score (t) = TF * IDF * UF
• Threshold values
– 30+ photographs
– Minimum 3 users
– Score > 1
• Final dataset: (tag, score, latitude, longitude)
![Page 12: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/12.jpg)
12yahooresearchberkeley.comRahul Nair
Talk Outline
• Mining information from geo-tagged photos
• World Explorer: Visualization
• ZoneTag: Creation
• Zurfer: Consumption
![Page 13: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/13.jpg)
13
DEMO
![Page 14: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/14.jpg)
14yahooresearchberkeley.comRahul Nair
Precomputation
• Divide the world into equal sized non-overlapping tiles
• Compute and store the tags for each tile
• Repeat for different zoom
levels
![Page 15: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/15.jpg)
15yahooresearchberkeley.comRahul Nair
Retrieval
• Find the tile level closest in size to the request area
• Select the tiles that fully cover the request area
• Return the tags that fall within the request area
![Page 16: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/16.jpg)
16yahooresearchberkeley.comRahul Nair
User Study
10 subjects
• 6 female, 4 male
• Ages 20-60
• Varying technical knowledge
• No geotagged photos of their own
![Page 17: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/17.jpg)
17yahooresearchberkeley.comRahul Nair
Experiment tasks
• Vacation recap
• San Francisco tour
• Explore a new city
![Page 18: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/18.jpg)
18yahooresearchberkeley.comRahul Nair
Recall
Reminded the subject about locations
• “It brings out memories”
• “Oh my God! This place has the best restaurants”
• “We wanted to see the Polynesian Cultural Center"
![Page 19: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/19.jpg)
19yahooresearchberkeley.comRahul Nair
Discovery
Participants discovered previously unknown locations and events
– “I’ve never heard of this festival”
– “There is car racing which I'd probably go see”
![Page 20: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/20.jpg)
20yahooresearchberkeley.comRahul Nair
Needle & Haystack
• Excellent visualization of the Haystack
• Hard to find specific information
– “Where was Culver City again?”
• No way to search
– “I guess what I’m looking for are bull fighting pictures”
![Page 21: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/21.jpg)
21yahooresearchberkeley.comRahul Nair
Other Responses
• Gets the “vibe” of a place
• Share with other people
• Tags did not always match the mental model of a location
• Wanted more tags
• Want more info about tags
![Page 22: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/22.jpg)
22yahooresearchberkeley.comRahul Nair
Conclusions
• Extract meaningful aggregate information from georeferenced data
• Allows users to explore locations in a new way
• Users like using the overview but also want the ability to search
![Page 23: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/23.jpg)
23yahooresearchberkeley.comRahul Nair
Future work
• Adding search capability
• Show photos in places with no tags
• Differentiate locations and events
• Apply to other types of georeferenced data
![Page 24: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/24.jpg)
24yahooresearchberkeley.comRahul Nair
Talk Outline
• Mining information from geo-tagged photos
• World Explorer: Visualization
• ZoneTag: Creation
• Zurfer: Consumption
![Page 25: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/25.jpg)
25yahooresearchberkeley.comRahul Nair
Current Mobile Experience
• Difficult to share (or even save!)
• Hard to find
– No context
– No semantic information
Current mobile experience?
![Page 26: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/26.jpg)
26yahooresearchberkeley.comRahul Nair
Current Mobile Experience
• Difficult to share (or even save!)
• Hard to find
– No context
– No semantic information
![Page 27: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/27.jpg)
27yahooresearchberkeley.comRahul Nair
ZoneTag Experience
• 2-click upload (same key!)
• Photo uploaded with location and time metadata
![Page 28: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/28.jpg)
28yahooresearchberkeley.comRahul Nair
Where does location come from?
• Bluetooth GPS (when available)
• User-contributed cell tower mapping
![Page 29: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/29.jpg)
29yahooresearchberkeley.comRahul Nair
ZoneTag Experience
• Tagging made easy
– Tag/annotate your photos from the phone
![Page 30: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/30.jpg)
30yahooresearchberkeley.comRahul Nair
Where do tags come from?
• Tags I used in this context (`home’)
• Tags my friends used in this context
• Tags other people used in this context (‘Ricoh’, ‘California Research Center’)
– E.g., TagMaps data
![Page 31: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/31.jpg)
31yahooresearchberkeley.comRahul Nair
Where do tags come from?
• Stuff around you:
– Yahoo! Local
– Upcoming.org
• Stuff from you (any RSS 2.0 feed):
– Calendar
– Favorite hangouts (Wayfaring, Plazes, Socialight)
![Page 32: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/32.jpg)
32yahooresearchberkeley.comRahul Nair
Example of Tagging
![Page 33: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/33.jpg)
34yahooresearchberkeley.comRahul Nair
Talk Outline
• Mining information from geo-tagged photos
• World Explorer: Visualization
• ZoneTag: Creation
• Zurfer: Consumption
![Page 34: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/34.jpg)
35yahooresearchberkeley.comRahul Nair
Flickr “geotagged”
20+ million images
Can we do mobile?
![Page 35: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/35.jpg)
36yahooresearchberkeley.comRahul Nair
Strengths of Mobile
• Personal
• Easy to access
• Networked
• Context aware
The ultimate photo wallet
![Page 36: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/36.jpg)
37yahooresearchberkeley.comRahul Nair
Design Goals
• Engagement and Discovery
– Spatial
– Social
• Allow customization
• Complete access to your photos
• Search and Filtering capability
![Page 37: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/37.jpg)
38yahooresearchberkeley.comRahul Nair
Channel Metaphor
• Each row is a single channel
• Navigate using 4 way
– Left & right to browse channel (infinite scroll)
– Up & down to change channels
![Page 38: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/38.jpg)
39yahooresearchberkeley.comRahul Nair
Detail view
• Enlarge photo
– Scroll through channel
• Photo details
• Add comments
• Add to favourites
![Page 39: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/39.jpg)
40yahooresearchberkeley.comRahul Nair
Nearby photos channel
• Photos from the users current location
• Local Highlights
• My Nearby
![Page 40: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/40.jpg)
41yahooresearchberkeley.comRahul Nair
Social Channels
• Contacts photos
– Expanded view
• Recent Activity
![Page 41: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/41.jpg)
42yahooresearchberkeley.comRahul Nair
My Stuff Channel
• My Photos
– By Location
– By Tag
• My Photo Wallet
• My Favourites
![Page 42: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/42.jpg)
43yahooresearchberkeley.comRahul Nair
Custom Channels
• Users can create channels to match their interests
– Tags
– Groups
– Location aware (dogs near me)
![Page 43: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/43.jpg)
44yahooresearchberkeley.comRahul Nair
Search and Filter
![Page 44: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/44.jpg)
45yahooresearchberkeley.comRahul Nair
Try it out
http://zurfer.research.yahoo.com
![Page 45: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/45.jpg)
46yahooresearchberkeley.comRahul Nair
Conclusions
• It is possible to extract information from georeferenced media
• Users like browsing the extracted data
• It can help users tag new media
• We hope it helps them browse on a mobile device
![Page 46: What's in a place? Adventures with Location-Aware Media](https://reader036.vdocuments.mx/reader036/viewer/2022070319/5582ef9ed8b42a26168b4a3d/html5/thumbnails/46.jpg)
47
Questions?
Rahul [email protected]
http://tagmaps.research.yahoo.comhttp://zonetag.research.yahoo.com http://zurfer.research.yahoo.com