collaborative image- based tagging and interactive mapping on tabletops presented by: glen whitaker...
Post on 22-Dec-2015
226 views
TRANSCRIPT
Collaborative Image-based Tagging and Interactive Mapping on TabletopsPresented By: Glen Whitaker
Supervisor: Judy Kay
Research Question
How to Support Tabletop interaction for organisation of Content?
Why is it important? Novelty of User Interface Organising is fundamental
Especially file systems
Constraints? Limited methods of input
Challenges? No established toolkits, primitives, UI design No guidelines Research - Cruiser Tabletop as exploratory framework
Refined Research Question
How to Support Tabletop interaction for organisation of Content using Tagging and Maps?
Why? Tagging text is similar to associating file name and
location to files in a file system Maps similar concept to the folder structure in a file
system
Sub-Problem: How to Support Tagging at Tabletop Adding Tags to Images
Linking Images to maps
Single text/image tags
Multiple Tags
Refinement: Elements of Tagging
Way to organise and find digital artefacts
Key interface tasks for tagging Defining tags Adding tags to objects Viewing tags Searching tags Deleting tags
How to achieve this on Tabletop Interfaces?
Approach 1: MyTags Inspired by TeamTag
Morris, M. R., A. Paepcke, T. Winograd, and J. Stamberger. "TeamTag: Exploring Centralized Versus Replicated Controls for Co-Located Tabletop Groupware." Proceedings of the SIGCHI conference on Human Factors in computing systems CHI: ACM Press, 2006. 1273 - 82.
Text-Based Tagging
Adding Text to Images
Adding Multiple Text to Images
Approach 2: MyMaps
Image-based Tagging in context of maps
Adding Images to Map Locations Placing image on a single map location/region
Adding Images to Multiple Map Locations Placing an image to multiple regions on map
Prototype
Defining Tags
Tags are predefined (externally in a file/database)
Tags represented by special objects
Adding Tags
MyTags MyMaps
Single Tag (1)
Move text to back of image
(1 gesture)
Dwell image on Map within region(s)
(1 gesture)
Multiple Tags (n)
Repeat single
(n gestures)
Same as single
(1 gesture)
Viewing Tags
MyTags Flip image over to view list of text
MyMaps Dwell on tag to view image
Viewing is same for single and multiple tags in both MyTags and MyMaps
Deleting Tags
Move tag to blackhole
Same for tags in both MyMaps and MyTags
Same for single and multiple tags
The Framework
Cruiser Images Blackhole (garbage collection object)
Tagging Primitives Adding Text Tags Adding Image based tags Adding Regions View Tags Delete Tags
Previous Elements
Previous Elements used by MyMaps: Blackhole (garbage collection object) Images (used to represent exhibits)
Audio can be associated with image Images can have other images placed on back
Images Blackhole
New Elements “New” elements of the Cruiser Tabletop
Map Resources Map Region Creation Tool “Pin Images” similar to pins on a notice board Image Region Identification
Region Tool
Pin Images
Map
Region
Back of Image
Text Object
Text Attachment
Architecture
Image
Original Sharepic Image
MyMaps Image Extensions
Pin
Text
Region Tag Component
Layout
Map Layout Map
Region Tool
Resource
Original Sharepic Resource
MyMaps Resource Extensions
Environment
Original Sharepic Environment
MyMaps Environment Extensions
Original Sharepic Blackhole
= Trent’s Cruiser Elements = New Elements
Evaluation Performance
Successful Tagging with MyTag Successful Tagging with MyMaps
Other usability elements Comparison of affordances Learnability Error Rate Error Recovery
Collocated Collaboration UI aspects Duplicated Maps UI Facilitates Collaboration
Evaluate Effects of Short term memory load Images Maps
Evaluation Design Qualitative Evaluations
6 Pairs (User A, User B)
Asked to design their own museum
Double cross over condition (TeamTag and MyMaps + MyTags)
User A and B given different sets of text attached to 10 images
Questionnaire for effective issues: Preference, attitudes, perspective
Evaluation Steps Tutorial
Task 1 Design virtual Museum
Task 2 Free-form virtual Museum
Tutorial of cross-over system
Repeat Tasks 1 and 2 for cross over system
Questionnaire
Double cross-over to control first system effect
Evaluate Collaboration and usability without design
limitations
Encourage Collaboration in Tagging Context
Participant Pair
TeamTag: Time (minutes)
MyMaps + MyTags: Time (minutes)
Notes Relating to Participant Background
1 (A, B) 27 33 Minimal Computing background
2 (A, B) 14 19 No prior contact with each other
3 (A, B) 13 19 Experienced Tabletop Users
4 (A, B) 16 12 Minimal Computing background
5 (A, B) 45 14 No prior contact with each user affected early communication. Mixture of participant with minimal computing background and experienced computer user
6 (A, B) 41 21 Mixture of participant with minimal computing background and experienced computer user
= Represents System used Second (always faster)
Average of System being used First (minutes) Average of System being used Second (minutes)
TeamTag MyMaps + MyTags TeamTag MyMaps + MyTags
34 23 18 16
= Represents System used Second = Preferred MethodX
User TeamTag MyMaps + MyTags
Pile Region Replicated Maps
Image Tags
Fixed Text
1:A X X X X X
1:B X X X X X
2:A X X X X
2:B X X X X X
3:A X X X X
3:B X X X X
4:A X X X X X
4:B X X X X X
5:A X X X
5:B X X X X
6:A X X X X
6:B X X X X
User TeamTag MyMaps + MyTags
Pile Region Replicated Maps
Image Tags
Fixed Text
1:A X X X X X
1:B X X X X X
2:A X X X X
2:B X X X X X
3:A X X X X
3:B X X X X
4:A X X X X X
4:B X X X X X
5:A X X X
5:B X X X X
6:A X X X X
6:B X X X X
= Represents System used Second = Preferred MethodX
10/12 Preferred Second System
2 preferred MyMaps even as first system
User TeamTag MyMaps + MyTags
Pile Region Replicated Maps
Image Tags
Fixed Text
1:A X X X X X
1:B X X X X X
2:A X X X X
2:B X X X X X
3:A X X X X
3:B X X X X
4:A X X X X X
4:B X X X X X
5:A X X X
5:B X X X X
6:A X X X X
6:B X X X X
= Represents System used Second = Preferred MethodX
2 People (1 Pair) liked bothOthers evenly split
Some commented regions suited task better and less
planning needed
User TeamTag MyMaps + MyTags
Pile Region Replicated Maps
Image Tags
Fixed Text
1:A X X X X X
1:B X X X X X
2:A X X X X
2:B X X X X X
3:A X X X X
3:B X X X X
4:A X X X X X
4:B X X X X X
5:A X X X
5:B X X X X
6:A X X X X
6:B X X X X= Represents System used Second = Preferred MethodX
5 People (2 Pairs + 1) liked but hardware limitations prevented effective use
User TeamTag MyMaps + MyTags
Pile Region Replicated Maps
Image Tags
Fixed Text
1:A X X X X X
1:B X X X X X
2:A X X X X
2:B X X X X X
3:A X X X X
3:B X X X X
4:A X X X X X
4:B X X X X X
5:A X X X
5:B X X X X
6:A X X X X
6:B X X X X
= Represents System used Second = Preferred MethodX
All participants liked Image Tagging and could use
comfortably
User TeamTag MyMaps + MyTags
Pile Region Replicated Maps
Image Tags
Fixed Text
1:A X X X X X
1:B X X X X X
2:A X X X X
2:B X X X X X
3:A X X X X
3:B X X X X
4:A X X X X X
4:B X X X X X
5:A X X X
5:B X X X X
6:A X X X X
6:B X X X X
= Represents System used Second = Preferred MethodX
Manly yes, participants lost tags creating objects
Pair 1 preferred flexibility to organise
Performance Results
Successful Tagging with MyTag Association of Text to objects performed with low
error rate Pairs completed tasks successfully using method
Successful Tagging with MyMaps Association of Images was performed with low error
rate Pairs completed tasks successfully using method
Other usability elements Swipe action for text tagging was liked and not really in the
design
Learnability Easy to learn tagging primitives
Error Rate/Recovery Low error rate for Swipe/movement text association technique
Errors were due to having multiple images flipped within path of movement
Low error rate for image tags on maps Errors were mainly a result of lack planning suitable location
Image Tags Errors fixed by deletion of reference point and further planning
Text Tags Errors fixed by deletion of attached text
Collocated Collaboration
Duplicate Maps Participants did not use replicated maps as expected
Hardware limitations - multiple use - affected this 3 out of 12 participants used second map as a
thumbnail view of the focus map
UI Facilitates Collaboration Participants shared text objects Participants collaboratively planned image tag
locations Participants collaboratively planned text tags for each
image
Effects on Short-Term Memory Load Images
Images are flipped in tagging, causing short-term memory load but avoiding confusion
Images could not be semi-transparent since it would conflict with other functions
Maps Dwelling needed to view images relating to tags image tag design reduced clutter and confusion about map
objects as the cost of the need to check what reference points represented
Participants were observed to have minor problems remembering tags with the small sample of images and text
Caveats
Hardware Mimio capturing technology used for Cruiser only
allows one interaction at a time
Software Objects are 2-Dimensional representations Text predefined in external file loaded at start up Precision of interaction depends on accuracy of
calibration
Future Work Handwriting for tag inputs
Backend database for efficiency (minimise I/O)
Personalised recommendations based on tags
Link tagging to file system – (Anthony Collins thesis)
Contribution
Significant progress in understanding how to support collocated collaborative planning at a tabletop
Insight into new gestures for image tagging and text tagging on tabletops
Swipe gesture discovered
No more slides. Click to continue.
Results
Was 6 pairs enough? Sample showed consistent patterns relating to Image
Tagging
Provided substantial usability, performance, collaboration and affordance results
More detailed and extensive evaluations required for further results
Background: Virtual Museum Implementation
Virtual Museums Digital Technology being used to display exhibits while preserving the
contents of an exhibit in digital format Augmented Reality and 3D Modelling advancements allow
photorealistic exhibits
Huang, C., C. Chen, and P. Chung. "Projects in VR: Tangible Photorealistic Virtual Museum." IEEE Computer Graphics and Applications: IEEE, 2005. pp 15 - 17.
Direct Touch Interfaces
DiamondTouch Tabletop Direct-Touch Based Interaction Unique identification based on circuits through chairs/pads
Other Touch based tables include: SmartSkin, Entertaible and Misto
Ryall, K., M.R. Morris, K. Everitt, C. Forlines, and C. Shen. "Experiences with and Observations of Direct-Touch Tabletops." First IEEE International Workshop on Horizontal Interactive Human-Computer Systems: IEEE, 2006.
Lumisight – Rear Projection Design
Lumisight Tabletop Projectors inside the table project different images in different
directions Number of users limited to number of projectors
Kakehi, Y., T. Hosomi, M. Iida, T. Naemura, and M. Matsushita. "Transparent Tabletop Interface for Multiple Users on Lumisight Table." First IEEE International Workshop on Horizontal Interactive Human-Computer Systems: IEEE, 2006.
TeamSearch TeamSearch
Use of special widgets to represent metadata shared by all users Search images through use of widgets Search through manually tagged images using widgets
Morris, M.R., A. Paepcke, and T. Winograd. "Teamsearch: Comparing Techniques for Co-Present Collaborative Search of Digital Media." First IEEE International Workshop on Horizontal Interactive Human-Computer Systems: IEEE, 2006. pp 97 - 104.
Widget
Metadata inside widgets
Personal Space
Query Objects
TeamSearch Added Tags
Tags in the original implementation were added manually to each image
Searching Users search the photos by placing query object onto the metadata
values Thumbnails of matching photo’s appear in that user’s personal space
Tags are Metadata that matches the contents of the widgets
TeamTag TeamTag
Presentation and association of metadata Centralised-control (TeamSearch approach) and Replicated-control
design
Morris, M. R., A. Paepcke, T. Winograd, and J. Stamberger. "TeamTag: Exploring Centralized Versus Replicated Controls for Co-Located Tabletop Groupware." Proceedings of the SIGCHI conference on Human Factors in computing systems CHI: ACM Press, 2006. 1273 - 82.
Centralised-Controls Design Replicated-Controls Design
TeamTag Centralised-controls design
Implementation of TeamSearch widgets Users attach metadata by tapping image,
then subsequently selecting a piece of metadata in a widget
Replicated-controls design Presents metadata categorised into
columns that are presented in front of each user
Users attach metadata by tapping image, then subsequently selecting a piece of metadata in the collection presented in front of them
TeamTag Added Tags
Tapping the image to be tagged and the subsequent piece of metadata selected by that user is attached
Multiple tags can be associated to images using this method
Tags are Defined sets of metadata
Viewing Tags Popup lists present what metadata is associated with an image Popup lists activated by a user touching an image with two fingers
Centralised Vs Replicated Evaluated in context of zoo, labelling each image with
appropriate metadata in groups of 4
19 out of 24 subjects preferred replicated metadata interface
Collaborative evaluation of replicated interface showed on average 1.24 users contributed to the tagging of an image
Group collaboration and communication levels presented no significant difference
Google Maps Google Maps
Interactively move around map of world Zoom in on satellite and road maps of areas around the world Identify businesses on the maps by providing business info to Google
to tag based on street address WikiMapia provides user defined region tagging
Google Maps Tour. Google. Available: http://www.google.com/help/maps/tour/. 10th of August 2006.
Google Maps Added Tags
WikiMapia – Drawing of a box on the map to indicate a region Google Maps – Entering of data with the location being labelled by the system
based on data inputted about a business
Searching Searches performed by comparing text inputted to the metadata that is
associated with a tag
Tags WikiMapia – regions are user defined boxes with a text tag identifying what the
region is Google Maps – identifies single addresses with business details associated
with that address (limited to USA, Japan, UK, China and Canada)
Point Vs Region WikiMapia allows user identification of regions Google maps allows identification of single addresses on maps
The Cruiser Cruiser Tabletop
Uses Mimio® to Capture Interaction Uses Pen stylus devices for interaction Number of users limited to Pen Numbers
MyMaps and current software on the cruiser can be implemented using DiamondTouch interfaces
Trent Apted’s First Year of PhD presentation on Photo sharing with Sharepic system and the Cruiser Tabletop
Map Resources Contribution
Explore how collaborative planning activities can be performed on tabletop interfaces
Provides a context for image based tagging
Transparency levels represent the interactive state
Maps are replicated for the number of users
Non-Interactive MapInteractive Map
Walls in Museum
Map With some Tags
Regions
Contribution Categorisation through areas on map objects Evaluate the effectiveness of planning through the identification
of regions
Using a region creation tool to identify these regions Image tagged to maps within regions inherit any text for
those regions
Region Creation Tool
Attached Regions
E.g. Greek
E.g. Roman
Image Tags Contribution
Evaluation of the effectiveness of image based tagging on collaborative interfaces
Evaluate how this assists in planning tasks on tabletop interfaces
Assigned by performing a gesture on the image to be attached while placed on top of the map
ImagesMap with Image Tags of 2
different Users
Text Association
Contribution Compare swipe (MyMaps) to tapping (TeamTag) Compare Map-based tagging and TeamTag approach
to tag Images
Attach text by dragging over the back of images and regions
Back of Image
Text Object
Text Attachment
Image Based Tagging Metadata stored in text format within a designated folder
Metadata stored for images and regions: Maps Regions Rotations Locations Text Scale
Each entry of a location in the tag files for each image and region represents a tag on a designated map
##LOCATION<map>: <x coordinate>, <y coordinate>, <z coordinate> <map>: <x coordinate>, <y coordinate>, <z coordinate>…
Image Based Tagging
Tags are represented by special “Pin Images” which are small (at the moment) blue spherical-like images that resemble a pin
Each “Pin Image” relies on a designated rotation entry and location entry
##ROTATION<radian value of rotation><radian value of rotation>…
##LOCATION<map>: <x coordinate>, <y coordinate>, <z coordinate> <map>: <x coordinate>, <y coordinate>, <z coordinate>…
Images that are linked to regions are also indirectly associated to the map that the region is linked to
More Video
Other Tabletop Research Benko, H., A.D. Wilson, and P. Baudisch. "Precise Selection Techniques for
Multi-Touch Screens." SIGCHI conference on Human Factors in computing systems ACM Press, 2006. pp 1263 - 72.
Ha, V., K. Inkpen, R. Mandryk, and T. Whalen. "Direct Intentions: The Effect of Input Devices on Collaboration around a Tabletop Display." First IEEE International Workshop on Horizontal Interactive Human-Computer Systems: IEEE, 2006.
Ryall, K., K. Everitt, C. Forlines, and C. Shen. "Multispace: Enabling Electronic Document Micro-Mobility in Table-Centric Mult-Devie Environments." First IEEE International Workshop on Horizontal Interactive Human-Computer Systems: IEEE, 2006.
Scott, S., M. Sheelagh, T. Carpendale, and K. Inkpen. "Territoriality in Collaborative Tabletop Workspaces " Proceedings of the ACM conference on Computer supported cooperative work ACM Press, 2004. 294 - 303.
Tang, A., M. Tory, B. Po, P. Neumann, and S. Carpendale. "Collaborative Coupling over Tabletop Displays." SIGCHI conference on Human Factors in computing systems CHI: ACM Press, 2006. pp 1181 - 90.
More Tags … Wikimapia. Available: http://www.wikimapia.org. 10th of August 2006.
Toyama, K., R. Logan, and A. Roseway. "Geographic Location Tags on Digital Images." Proceedings of the eleventh ACM international conference on Multimedia ACM Press, 2003. pp 156 - 66.
Marlow, C., M. Naaman, D. Boyd, and M. Davis. "Ht06, Tagging Paper, Taxonomy, Flickr, Academic Article, to Read." The seventeenth conference on Hypertext and hypermedia ACM Press, 2006. pp 31 - 40.
Del.Icio.Us. Available: http://del.icio.us/. 3rd of September 2006.
Google Maps. Google. Available: http://maps.google.com/. 10th of August 2006.
Morris, M. R., A. Paepcke, T. Winograd, and J. Stamberger. "Teamtag: Exploring Centralized Versus Replicated Controls for Co-Located Tabletop Groupware." Proceedings of the SIGCHI conference on Human Factors in computing systems CHI: ACM Press, 2006. 1273 - 82.
References Koshizuka, N., and K. Sakamura. "Tokyo University Digital Museum." International
Conference on Digital Libraries: Research and Practice Kyoto, 2000. pp 85 - 92.
WikiMapia. Google. Available: http://www.wikimapia.org. 10th of August 2006.
Kakehi, Y., T. Hosomi, M. Iida, T. Naemura, and M. Matsushita. "Transparent Tabletop Interface for Multiple Users on Lumisight Table." First IEEE International Workshop on Horizontal Interactive Human-Computer Systems: IEEE, 2006.
Huang, C., C. Chen, and P. Chung. "Projects in VR: Tangible Photorealistic Virtual Museum." IEEE Computer Graphics and Applications: IEEE, 2005. pp 15 - 17.
Morris, M. R., A. Paepcke, T. Winograd, and J. Stamberger. "Teamtag: Exploring Centralized Versus Replicated Controls for Co-Located Tabletop Groupware." Proceedings of the SIGCHI conference on Human Factors in computing systems CHI: ACM Press, 2006. 1273 - 82.
References Morris, M.R., A. Paepcke, and T. Winograd. "Teamsearch: Comparing Techniques
for Co-Present Collaborative Search of Digital Media." First IEEE International Workshop on Horizontal Interactive Human-Computer Systems: IEEE, 2006. pp 97 - 104.
Trent Apted’s First Year of PhD presentation on Photo sharing with Sharepic system and the Cruiser Tabletop
Ryall, K., M.R. Morris, K. Everitt, C. Forlines, and C. Shen. "Experiences with and Observations of Direct-Touch Tabletops." First IEEE International Workshop on Horizontal Interactive Human-Computer Systems: IEEE, 2006.
Google Maps Tour. Google. Available: http://www.google.com/help/maps/tour/. 10th of August 2006.