on the utility of tags for search and navigation in online information systems

Graz University of Technology

1

. Christoph Trattner Rigorosum 11.10.2012

On the Utility of Tags for Search and Navigation in Online Information Systems

Christoph Trattner

Knowledge Management Institute

Graz University of Technology, Austria


2


What will this talk about

Tagging Systems and in particular about

Tags/tag clouds and

their usefulness for the task of search and navigation


3


Definitions

Tag =

A short string, term or word that describes or categorizes an online resource and that is applied by a person or a set people

Tagging System =

An online information system that allows the users to apply tags to resources of the system


4


What was the motivation of my work?


5


Motivation?

What I recognized when I started my PhD work 3 years ago was the fact that a lot of modern online information systems used tagging functionality to categorize or describe content

and…

to build simple user interfaces on the top-of this light-weight meta-data structures


6



7


Tags


8


InterestinglyThere is a lot of work,

Analysis of Tagging Systems

[Hammond et al.] [Golder and Huberman] [Marlow et al.] [Halphin et al.] [Shen and Wu]

Tags vs. Keywords vs. Named Entities (semantic/structure)

[Krause et al.] [Benz et al.] [Heymann et al.]

Tagging Motivation and Behavior

[Heckner et al.] [Ames and Naaman] [Strohmaier et al.]

[Körner et al.]

Tag Cloud Construction & Visualization

[Montero and Solana] [Kautz et al.] [Rivadeneira et al.] [Kaser and Lemire] [Seifert et al.]

Utility of Tag Clouds for Search Result Summarization

[Kuo et al.] [Koutrika et al.] [Sinclair et al.]

There is hardly any research that investigates the usefulness

of tags or tag clouds for the task of search and navigation

in tagging systems


9


Problem Statement

The problem we are facing in this dissertation is the lack of knowledge about the usefulness and the efficiency of tags and corresponding state-of-the-art tag-constructs such as tag clouds for the task of search and navigation in tagging systems.


10


Research Questions

RQ1: To what extent are tags/tag clouds useful for (efficient) navigation in tagging systems?

RQ2: To what extent are tags/tag clouds useful for search?

RQ3: To what extent are tags/tag clouds more useful/efficient for search/navigation than other tag-alike meta-data such as keywords or search query-terms?

RQ4: To what extent can we build better tag-based browsing constructs that support efficient search/navigation in tagging systems?


11


Ok, lets start….


12


Research Question 1:

To what extent are tags/tag clouds useful for (efficient) navigation in tagging systems?

Helic, D., Trattner, C., Strohmaier, M. and Andrews, K. 2010. On

the Navigability of Social Tagging Systems. In Proceedings of the Second

IEEE International Conference on Social Computing (SocialCom

2010), Minneapolis, Minnesota, USA, pp. 161-168.


13


Modeling tagging systems as graphs

Resources (= Text Documents, Images, URLs)

Tags

To answer the question to what extent tags/tag clouds are useful for navigationin tagging systems we modeled tagging systems as bipartite graphs


14


Defining Navigability

A network is navigable iff:There is a short path between all or almost all pairs of

nodes in the network. [Kleinberg 1999]

Formally:1. There exists a giant component (> 90%)2. The effective diameter is low (bounded by log n)

J. Kleinberg. The small-world phenomenon: An algorithmic perspective. Proc. 32nd ACM Symposium on Theory of Computing, 2000. Also appears as Cornell Computer Science Technical Report 99-1776 (October 1999)

http://www.cs.cornell.edu/home/kleinber/swn.pdf


15


Are tags useful for navigation?

Results:

In general tags form networks which are navigable

Austria-Forum: 32,245 annotations, 12,837 resourcesBibSonomy: 916,495 annotations, 235,339 resourcesCiteULike: 6,328,021 annotations, 1,697,365 resources


16


Are tags useful for efficient navigation?

.

Tagging networks are navigable power-law networks. For power law networks, efficient sub-linear decentralised navigation algorithms exist.

Results:

In general tags form networks which are also efficiently navigable


17


But how about tag clouds?

Tag Cloud Size ntopN resources

(topN most common algorithm)

Pagination of resources / tagk resources shown / page

(reverse chronological ordering)


18


Are tag clouds useful for navigation?

.

Limiting the tag cloud size n to practically feasible sizes (e.g. 5, 10, or more) does not influence navigability (this is not very surprising).

BUT: Limiting the out-degree of high frequency tags k (e.g. through pagination with resources sorted in reverse-chronological order) leaves the network vulnerable to fragmentation. This destroys navigability of prevalent approaches to tag clouds.

Pagination

Tag Cloud Size

Results:

In general tag clouds do not provide the possibility to navigate to all resources in a tagging system


19



To what extent are tags/tag clouds useful for search?

Trattner, C., Lin, Y., Parra, D., Yue, Z., Real, W. and Brusilovsky, P.

2012. Evaluating Tag-Based Information Access in Image Collections.

In Proceedings of the 23rd ACM Conference on Hypertext and Social

Media (HT 2012), ACM, New York, NY, USA, pp. 113-122.


20


Methodolgy

A controlled user study with 24 participants

With three different types of search interfaces

Baseline Tag Cloud Faceted Tag Cloud

1 2 3


21


Dataset

~ 2,000 images ~ 4,200 tags ~ 16,000 tag assignments

Interesting Fact:

Tags were generated by ~100 users from Amazon Mechanical Turk


22


Evaluation: Look-up Task

Look-up task


23


Evaluation: Exploratory Search Task

Exploratroy search task


24


Results: Performance

Look-up: no sign. differences between interfaces

Exploratory:- Tag Cloud Interface out-performs baseline- Faceted Tag Cloud Interface almost as slow as baseline

Question: What interface performs best?Variables: • Total Actions• Search Time

1 2 3

Results:

The Tag cloud interface significantly outperforms the baseline (no-tag) interface


25


Results: Preference and RatingQuestion: What was the preference of the users?

• Post-questionair was handed out to the subjects with overall 7 questions.

Question: How are the interfaces rated?

Scale: 1 = very bad….5=very good

Results:

The Tag cloud interfaces are significantly higher rated than the non-tag interface

No tags tags


26



To what extent are tags/tag clouds more useful/efficient for search/navigation than other tag-alike meta-data such as keywords or search query-terms?

Trattner, C. 2011. Linking Related Content in Web Encyclopedias with search query tag clouds. In the International Journal on WWW/Internet, Volume 9, Issue 2 (IJWI), pp. 33-55.

Helic, D., Körner, C., Granitzer, M., Strohmaier, M. and Trattner, C. 2012. Navigational Efficiency of Broad vs. Narrow Folksonomies. In Proceedings of the 23rd ACM Conference on Hypertext and Social Media (HT 2012), ACM, New York, NY, USA, pp. 63-72.


27


Example: Austria-Forum

In Austria-Forum tags/tag clouds are used to link related content

Since the tagging system is in a early adopting phase search query terms are used


28


Are query terms more navigable than tags?

Results:Both user tag and query tag networks show a large connected componentBoth show an ED that is bounded by log(N)

Results:

On a network-theoretic level we find that the AF user tags and query tags are efficiently navigable


29


And how efficient are they for humans?

Tag hierarchy Tag (Cloud) network

To that end, we implemented a decentralized search algorithm that simulates human-like tag-based navigation by inducing a hierarchy out of the tag-resource network [Helic 2011] [Trattner 2012].

Trattner, C., Singer, P., Helic, D. and Strohmaier, M.: Exploring the Differences and Similarities of Hierarchical Decentralized Search and Human Navigation in Information-networks, In Proceedings of the 12th International Conference on Knowledge Management and Knowledge Technologies, ACM, New York, NY, USA, 2012.

Helic, D., Strohmaier, M., Trattner, C., Muhr M. and Lermann, K.:Pragmatic Evaluation of Folksonomies, In Proceedings of the 20th international conference on World wide web, ACM, New York, NY, USA, 417-426, 2011.


30


What are the results?

Results:

From simulations we find that the query tags are better suited for navigation than user tags

Additionally to this a user study was conducted to determine the quality of the query tags, showingno sign. difference.


31


We

Keywords

Tags

Example: Mendeley

Keyword = A short term or string (typical controlled vocabulary) assigned by a single user


32


Tags

Are keywords more navigable than tags?

Keywords

Results: Our Greedy Navigator (= Simulator) needs on average 1-click more with keywords to reach the target node than with tags

Results:

With simulations we find that tags are more efficient for navigation than keywords

#hops success rate #hops success rate

Stretch= #hops/shortest path


33


“Since we observed that tagging systems only support efficient navigation through tags if no user interface limitations are considered, we had the idea to invent a number of new approaches that support more efficient navigation with tags.”


34



To what extent can we build better tag-based browsing constructs that support efficient search/navigation in tagging systems?

Trattner, C., Helic, D. and Strohmaier, M. 2011. On the Construction of Efficiently Navigable Tag Clouds Using Knowledge From Structured Web Content. In the Journal of Universal Computer Science (JUCS), Volume 17, Issue 4, pp. 565-582.

Trattner, C. 2011. Improving the Navigability of Tagging Systems with Hierarchically Constructed Resource Lists: A Comparative Study. In Proceedings of the 33rd International Conference on Information Technology Interfaces (ITI 2011), IEEE, Cavtat / Dubrovnik, Croatia, pp. 173-178.

Trattner, C., Körner, C. and Helic, D. 2011. Enhancing the Navigability of Social Tagging Systems with Tag Taxonomies. In Proceedings of the 11th International Conference on Knowledge Management and Knowledge Technologies (I-Know 2011). ACM, New York, NY, USA, pp. 18:1-18:8.

Trattner, C. 2011. Improving the Navigability of Tagging Systems with Hierarchically Constructed Resource Lists and Tag Trails. In the Journal of Computing and Information Technology (CIT), Volume 19, Issue 3, pp. 155-167.


35


How can we enhance Tag Cloud navigability?


36


Through dynamic resource list construction!

Idea


37


Approach

Instead of calculating the resource list statically, we calculate the resource list in a dynamic and resource-specific manner=> On each click on a particular tag a different resource list is generated

Link1Link2Link4Link5


Tag Tag

Resource x Resource y


Tag Tag

Resource x Resource y


Static Resource List Construction Dynamic Resource List Construction


38


Approach: Hierarchical Resource List Construction


39


Results

Random

Results: Only the random approach and the hierarchicalresource list calculation approach show a large connected component

Hierarchical

Giant Component

Similarity


40


Results

Hierarchically Constructed Resource List

Simulations

User Study

Results:

We find tag clouds calculating the resource list in a hierarchical manner are better suited for navigation


41


…ok lets come to an end


42


Summary of Contributions

1. The review of the utility of tags for the task of search and navigation in tagging systems

2. The navigational review of tags compared to other tag-alike meta-data structures such as keywords and search query terms

3. The introduction of (a number) of new approach(es) to support more efficient tag-based navigation in tagging systems


43


Thank you!

Christoph Trattner

Email: [email protected]: www.christophtrattner.info

Twitter: @ctrattner

Sponsors:

on the utility of tags for search and navigation in online information systems

Documents

christoph trattner rigorosum

christoph trattnerrigorosum

utility of tag clouds

tagstag clouds useful

extent tagstag clouds

tagging functionality

abouttagging systems

usefulness tagging motivation