Web Search Results Visualization: Evaluation of Two Semantic Search Engines

Kalliopi Kontiza, k.kontiza.12@ucl.ac.ukAntonis Bikakis, a.bikakis@ucl.ac.uk

University College London, Department of Information Studies

Overview

● Semantic search engines improve the accuracy of search results:

- by understanding the meaning and context of terms as they appear in web documents,

- by using semantics to represent and process the user’s queries and the web data.

● Other parameters that define the quality of a search engine:

- its performance,

-its usability,

-the presentation of the search results

Overview

“Whether and how do semantic search engines improve

the visualization of search results, enhancing the search

experience? “

Structure of the Presentation

● Methodology

● Background information (InfoVis)

○ 1. Analytical Inspection

● Experiment

○ 2. The User Evaluation

● Results of the User Evaluation

● Discussion

Methodology

● An analytical Inspection

area of heuristic evaluation

‘the Visual Information-Seeking Mantra’ , Shneiderman (1996)

● A user-oriented evaluation study

● Interactive Information Retrieval (IIR) systems:

semantic search engines Sig.ma and Kngine

1. Background Information

Information Visualization

- Works as umbrella for all kinds of visualizations

- Best applied for exploratory tasks

- Ultimate purpose : amplify cognition

- Requires well formed data

2. Analytical Inspection

1. Overview Task-domain 2. Details on demand information3. Filter out/Highlight actions supported 4. Relate by an information 5. History visualization system, that6. Export users wish to perform

➔ Layout of the SUIs: Control (ie more), Input (ie search box) Features of SERP

Personalised (ie move content) interface Informational (ie result item)

2. Analytical Inspection

(Questionnaire videos)

3. Design & Set up of the User Evaluation

A. Variables a. Dependent 1. Overview i. Task domain information actions 2. Details on demand

3. Filtering outii. User Satisfaction 4. Relate

5. History b. Independent 6. Export

Predefined queries: a) Web Transactional Navigational

b) Informational Factual Source

3. Design & Set up of the User Evaluation

B. Questionnaire

- Online, closed-type questions, 5 point Likert, - Pre assigned queries presented in playlist of videos

- Sections: A. IntroductionB. EvaluationC. About

C. Sample- 83 participants

4. Results of the User Evaluation

● 55% male, 45% female

● 34% 18-26 age group, 51% 27-33 age group, 11% 34-40 age group

● 67% had used more than one search engines

● 86% rated their search skills with 4 and 5 on 5 point Likert scale

The comparative presentation of user ratings for the visualization of the task domain information action criteria

4. Results of the User Evaluation

● Informational tasks received 71%, 61%

● Visualization was ranked 4th, 73% graded it with 4 and 5 on 5 point Likert scale

● User-satisfaction perceived acceptance:o good satisfaction for history

and export but o more expectations from

overview and details on demand

5. Discussion

-Different perspective to view data

-Non linear and dynamic visualization favoured

Q1. The visualization of the search results in semantic search engines improves the understanding of data and supports the user in assessing search results.

5. Discussion

- Visualization was ranked important in tasks as Overview, Details on demand, Filter out, Relate

-Careful consideration regarding additional visual representations

Q2. Semantic search engines make more effective use of visualization in displaying search results providing a better user experience.

5. Discussion

Q3. Semantics improve the visualization of search results.

-User can filter out due to the semantically organised data in properties and values

-The visualization of that task receives high preference amongst users

5. Discussion

Q4. The visualization of search results in semantic search engines provides a better search and thus user’s satisfaction.

-Users satisfied in general with the visualizations of the semantic search engines

-Visualization of search results plays a significant role in shifting users searching behaviour

Conclusions

● While visualization methods used by semantic search engines improves user

understanding of the results, the extent to which visualization methods are used

in such search engines can be improved even more.

● User experience rated positively but user satisfaction not accomplished in all

cases

Further questions to investigate

A more in-depth analysis needs to be performed on the collected data:● Are there any differences in the results of the user evaluation for the different

types of queries, considering the type of data that is searched or the complexity of the query?

● Is there any correlation, for example, between the user characteristics and the obtained data?

● Could a standardized cognitive and ability test help us further investigate the relationship between information visualization in semantic search engines and knowledge visualization ?

Thank you for your attention