nl#graphs:+a+hybrid+approach+toward+ …nl#graphs:+a+hybrid+approach+toward+...
TRANSCRIPT
![Page 1: NL#Graphs:+A+Hybrid+Approach+toward+ …NL#Graphs:+A+Hybrid+Approach+toward+ Interac9vely+Querying+Seman9c+Data Khadija Elbedweihy+ Suvodeep+Mazumdar+ StuartN.+Wrigley+ Fabio+Ciravegna](https://reader035.vdocuments.mx/reader035/viewer/2022063001/5f1c6d137e60a701484d03d3/html5/thumbnails/1.jpg)
NL-‐Graphs: A Hybrid Approach toward Interac9vely Querying Seman9c Data
Khadija Elbedweihy Suvodeep Mazumdar Stuart N. Wrigley Fabio Ciravegna
OAK Group University of Sheffield
Sheffield, UK
k.Elbedweihy | s.mazumdar | s.wrigley | f.ciravegna | @ shef.ac.uk ovus00
![Page 2: NL#Graphs:+A+Hybrid+Approach+toward+ …NL#Graphs:+A+Hybrid+Approach+toward+ Interac9vely+Querying+Seman9c+Data Khadija Elbedweihy+ Suvodeep+Mazumdar+ StuartN.+Wrigley+ Fabio+Ciravegna](https://reader035.vdocuments.mx/reader035/viewer/2022063001/5f1c6d137e60a701484d03d3/html5/thumbnails/2.jpg)
Users and User Experience
• Seman9c Search System • Assist users sa9sfy their informa9on needs
• User Sa9sfac9on – Query approach – Performance of Search system – Presenta9on of results – ….
• Usability and User Experience largely ignored – Need to focus on users
![Page 3: NL#Graphs:+A+Hybrid+Approach+toward+ …NL#Graphs:+A+Hybrid+Approach+toward+ Interac9vely+Querying+Seman9c+Data Khadija Elbedweihy+ Suvodeep+Mazumdar+ StuartN.+Wrigley+ Fabio+Ciravegna](https://reader035.vdocuments.mx/reader035/viewer/2022063001/5f1c6d137e60a701484d03d3/html5/thumbnails/3.jpg)
Query Approaches
• Types • Interac9on mechanisms • Formalisms (Natural language – Forms – Formal Query)
• O[en Task or applica9on Specific – Keyword-‐based approach to quickly iden9fy possible relevant
instances – Formal query for more complex tasks (nega9on, group-‐by etc.)
• Most SW applica9ons focus on one approach
![Page 4: NL#Graphs:+A+Hybrid+Approach+toward+ …NL#Graphs:+A+Hybrid+Approach+toward+ Interac9vely+Querying+Seman9c+Data Khadija Elbedweihy+ Suvodeep+Mazumdar+ StuartN.+Wrigley+ Fabio+Ciravegna](https://reader035.vdocuments.mx/reader035/viewer/2022063001/5f1c6d137e60a701484d03d3/html5/thumbnails/4.jpg)
Esther Kaufmann and Abraham Bernstein. Evalua9ng the usability of natural language query languages and interfaces to seman9c web knowledge bases. Journal of Web Seman/cs, 8(4):377–393, 2010.
Spectrum of Query Approaches
4
![Page 5: NL#Graphs:+A+Hybrid+Approach+toward+ …NL#Graphs:+A+Hybrid+Approach+toward+ Interac9vely+Querying+Seman9c+Data Khadija Elbedweihy+ Suvodeep+Mazumdar+ StuartN.+Wrigley+ Fabio+Ciravegna](https://reader035.vdocuments.mx/reader035/viewer/2022063001/5f1c6d137e60a701484d03d3/html5/thumbnails/5.jpg)
Issues + Mo9va9ons • Comparison of Query Approaches (Expert and Casual) [2]
§ NL Approach + Prior experience with Google-‐like approaches + Easy and quick to write ques9ons/keywords - Less support for Complex queries
§ View Based Approach + Good support in complex queries + Visualising the search space - Time and effort required formula9ng queries
• Learnability study with Graph based approach (Experts) [3] + Improvement in user sa9sfac9on and performance over 9me - S9ll needed 9me and effort to formulate queries - Extensive use of keyword-‐search for concepts
[2] K. Elbedweihy, S. N. Wrigley, and F. Ciravegna. Evalua9ng Seman9c Search Query Approaches with Expert and Casual Users. In Proceedings of the Evalua9ons and Experiments Track, ISWC, 2012. [3] S.Mazumdar, D.Petrelli, K. Elbedweihy, V. Lanfranchi, and F. Ciravegna. Affec9ve Graphs: The visual appeal of linked data. Seman9c web : interoperability, usability, applicability, 2013. In Press.
![Page 6: NL#Graphs:+A+Hybrid+Approach+toward+ …NL#Graphs:+A+Hybrid+Approach+toward+ Interac9vely+Querying+Seman9c+Data Khadija Elbedweihy+ Suvodeep+Mazumdar+ StuartN.+Wrigley+ Fabio+Ciravegna](https://reader035.vdocuments.mx/reader035/viewer/2022063001/5f1c6d137e60a701484d03d3/html5/thumbnails/6.jpg)
Affec9ve Graphs
![Page 7: NL#Graphs:+A+Hybrid+Approach+toward+ …NL#Graphs:+A+Hybrid+Approach+toward+ Interac9vely+Querying+Seman9c+Data Khadija Elbedweihy+ Suvodeep+Mazumdar+ StuartN.+Wrigley+ Fabio+Ciravegna](https://reader035.vdocuments.mx/reader035/viewer/2022063001/5f1c6d137e60a701484d03d3/html5/thumbnails/7.jpg)
Affec9ve Graphs
• Demo
![Page 8: NL#Graphs:+A+Hybrid+Approach+toward+ …NL#Graphs:+A+Hybrid+Approach+toward+ Interac9vely+Querying+Seman9c+Data Khadija Elbedweihy+ Suvodeep+Mazumdar+ StuartN.+Wrigley+ Fabio+Ciravegna](https://reader035.vdocuments.mx/reader035/viewer/2022063001/5f1c6d137e60a701484d03d3/html5/thumbnails/8.jpg)
HybridGraphs
• NL [4] + Graph [3] = Hybrid Query Approach • Builds on Advantages of either approach
• Quick access to relevant data • Easy and visual means to formulate complex queries • Visually inspect search space
• Alleviates the disadvantages • Lesser 9me taken to build complex queries
[3] S.Mazumdar, D.Petrelli, K. Elbedweihy, V. Lanfranchi, and F. Ciravegna. Affec9ve Graphs: The visual appeal of linked data. Seman9c web : interoperability, usability, applicability, 2013. In Press. [4] K. Elbedweihy, S.N. Wrigley, F. Ciravegna, and Z. Zhang (2013). Using BabelNet in bridging the gap between natural language queries and linked data concepts. NLP & DBpedia Workshop, 12th Interna9onal Seman9c Web Conference (ISWC2013), 21-‐25 October, Sydney, Australia.
![Page 9: NL#Graphs:+A+Hybrid+Approach+toward+ …NL#Graphs:+A+Hybrid+Approach+toward+ Interac9vely+Querying+Seman9c+Data Khadija Elbedweihy+ Suvodeep+Mazumdar+ StuartN.+Wrigley+ Fabio+Ciravegna](https://reader035.vdocuments.mx/reader035/viewer/2022063001/5f1c6d137e60a701484d03d3/html5/thumbnails/9.jpg)
Hybrid Approach
![Page 10: NL#Graphs:+A+Hybrid+Approach+toward+ …NL#Graphs:+A+Hybrid+Approach+toward+ Interac9vely+Querying+Seman9c+Data Khadija Elbedweihy+ Suvodeep+Mazumdar+ StuartN.+Wrigley+ Fabio+Ciravegna](https://reader035.vdocuments.mx/reader035/viewer/2022063001/5f1c6d137e60a701484d03d3/html5/thumbnails/10.jpg)
Evalua9on • Previous evalua9ons on NL component and Affec9ve Graphs [3,4]
(performance, usability, effec9veness etc.) • Focus on hybrid approach as a new query mechanism and the
support provided • Methodology
– Expert (SW experts in Computer Science + So[ware development firm) and Casual users (staff and students at the University of Sheffield)
– 24 subjects (12 + 12) – Aged between 18 and 53, mean age 31 years – Short demonstra9on of a scenario based example – Familiarisa9on with the system
[3] S.Mazumdar, D.Petrelli, K. Elbedweihy, V. Lanfranchi, and F. Ciravegna. Affec9ve Graphs: The visual appeal of linked data. Seman9c web : interoperability, usability, applicability, 2013. In Press. [4] K. Elbedweihy, S.N. Wrigley, F. Ciravegna, and Z. Zhang (2013). Using BabelNet in bridging the gap between natural language queries and linked data concepts. NLP & DBpedia Workshop, 12th Interna9onal Seman9c Web Conference (ISWC2013), 21-‐25 October, Sydney, Australia.
![Page 11: NL#Graphs:+A+Hybrid+Approach+toward+ …NL#Graphs:+A+Hybrid+Approach+toward+ Interac9vely+Querying+Seman9c+Data Khadija Elbedweihy+ Suvodeep+Mazumdar+ StuartN.+Wrigley+ Fabio+Ciravegna](https://reader035.vdocuments.mx/reader035/viewer/2022063001/5f1c6d137e60a701484d03d3/html5/thumbnails/11.jpg)
Evalua9on Tasks
• 5 ques9ons derived from QALD dataset [5] • Different levels of difficulty
– 1. When was Capcom founded? – 2. What did Bruce Carver die from? – 3. Who was the wife of U.S. president Lincoln? – 4. Give me all ci9es in Alaska with more than 10000 inhabitants. – 5. Show me all songs from Bruce Springsteen released between 1980
and 1990.
[5] V. Lopez, C. Unger, P. Cimiano, and E. Moqa. Evalua9ng ques9on answering over linked data. Web Seman9cs: Science, Services and Agents on the World Wide Web, 2013.
![Page 12: NL#Graphs:+A+Hybrid+Approach+toward+ …NL#Graphs:+A+Hybrid+Approach+toward+ Interac9vely+Querying+Seman9c+Data Khadija Elbedweihy+ Suvodeep+Mazumdar+ StuartN.+Wrigley+ Fabio+Ciravegna](https://reader035.vdocuments.mx/reader035/viewer/2022063001/5f1c6d137e60a701484d03d3/html5/thumbnails/12.jpg)
Evalua9on Analysis
• Objec9ve and Subjec9ve data collected • Input Time, number of aqempts, Success rate • Post evalua9on ques9onnaires – System Usability Scale (SUS),
Extended ques9onnaire – The query construc9on process was X. (5-‐point Likert scale, ranging
from Laborious to Effortless) – What did you like about the hybrid approach as a mechanism for
expressing your query? and why? – What things you didn't like about the hybrid approach as a mechanism
for expressing your query? and why? • Aesthe9cs ques9onnaire
![Page 13: NL#Graphs:+A+Hybrid+Approach+toward+ …NL#Graphs:+A+Hybrid+Approach+toward+ Interac9vely+Querying+Seman9c+Data Khadija Elbedweihy+ Suvodeep+Mazumdar+ StuartN.+Wrigley+ Fabio+Ciravegna](https://reader035.vdocuments.mx/reader035/viewer/2022063001/5f1c6d137e60a701484d03d3/html5/thumbnails/13.jpg)
Results
• Overall, experts (73.75) liked NL Graphs more than casuals (61.25) – Graph based approach more complicated – Experts more familiar with Seman9c Web and graph data
• Success rate 100% (all users were able to successfully complete the tasks) mostly on one aqempt
• Query Construc9on process scored effortless • Most users commented posi9vely on the usability and support
provided by the hybrid approach • Four users (1e + 3c)
– Complicated and non-‐intui9ve – Why are there more steps required to build queries than with Google? – “It seemed an extra step to get to your answer rather than just typing
in a search and it appearing in results.”
![Page 14: NL#Graphs:+A+Hybrid+Approach+toward+ …NL#Graphs:+A+Hybrid+Approach+toward+ Interac9vely+Querying+Seman9c+Data Khadija Elbedweihy+ Suvodeep+Mazumdar+ StuartN.+Wrigley+ Fabio+Ciravegna](https://reader035.vdocuments.mx/reader035/viewer/2022063001/5f1c6d137e60a701484d03d3/html5/thumbnails/14.jpg)
Results Contd.
• More assistance needed/desired • Need for more Training
• Experts took longer (94.48s) to formulate queries than casual (76.88s) users – Follow logic and prior understanding of SW concepts – Increased interest in the interface, trying out new features, exploring
technical aspects (e.g. SPARQL query generated)
![Page 15: NL#Graphs:+A+Hybrid+Approach+toward+ …NL#Graphs:+A+Hybrid+Approach+toward+ Interac9vely+Querying+Seman9c+Data Khadija Elbedweihy+ Suvodeep+Mazumdar+ StuartN.+Wrigley+ Fabio+Ciravegna](https://reader035.vdocuments.mx/reader035/viewer/2022063001/5f1c6d137e60a701484d03d3/html5/thumbnails/15.jpg)
Results Contd.
![Page 16: NL#Graphs:+A+Hybrid+Approach+toward+ …NL#Graphs:+A+Hybrid+Approach+toward+ Interac9vely+Querying+Seman9c+Data Khadija Elbedweihy+ Suvodeep+Mazumdar+ StuartN.+Wrigley+ Fabio+Ciravegna](https://reader035.vdocuments.mx/reader035/viewer/2022063001/5f1c6d137e60a701484d03d3/html5/thumbnails/16.jpg)
Results Contd.
• Q4 – Give me all ci9es in Alaska with more than 10000 inhabitants
– Use of terms like “Alaskan” – DbPedia proper9es such as isPartOf – Numerical constraints >10000
• Q5 -‐ Show me all songs from Bruce Springsteen released between 1980 and 1990 – Sezng dates as a constraint is a manual process
![Page 17: NL#Graphs:+A+Hybrid+Approach+toward+ …NL#Graphs:+A+Hybrid+Approach+toward+ Interac9vely+Querying+Seman9c+Data Khadija Elbedweihy+ Suvodeep+Mazumdar+ StuartN.+Wrigley+ Fabio+Ciravegna](https://reader035.vdocuments.mx/reader035/viewer/2022063001/5f1c6d137e60a701484d03d3/html5/thumbnails/17.jpg)
Summary
• Hybrid approach to querying Linked Data • It works ! J • High SUS scores indicate usability and usefulness • Future work
– Several re-‐designs iden9fied/suggested in the evalua9on – A longer term study directly comparing Hybrid approach to other
approaches in iden9cal condi9ons – Study with larger set of user groups – Improving NL system
[3] S.Mazumdar, D.Petrelli, K. Elbedweihy, V. Lanfranchi, and F. Ciravegna. Affec9ve Graphs: The visual appeal of linked data. Seman9c web : interoperability, usability, applicability, 2013
Aesthe9c design in Seman9c Web Applica9ons [3]
![Page 18: NL#Graphs:+A+Hybrid+Approach+toward+ …NL#Graphs:+A+Hybrid+Approach+toward+ Interac9vely+Querying+Seman9c+Data Khadija Elbedweihy+ Suvodeep+Mazumdar+ StuartN.+Wrigley+ Fabio+Ciravegna](https://reader035.vdocuments.mx/reader035/viewer/2022063001/5f1c6d137e60a701484d03d3/html5/thumbnails/18.jpg)
Thank You
• Ques9ons ?
![Page 19: NL#Graphs:+A+Hybrid+Approach+toward+ …NL#Graphs:+A+Hybrid+Approach+toward+ Interac9vely+Querying+Seman9c+Data Khadija Elbedweihy+ Suvodeep+Mazumdar+ StuartN.+Wrigley+ Fabio+Ciravegna](https://reader035.vdocuments.mx/reader035/viewer/2022063001/5f1c6d137e60a701484d03d3/html5/thumbnails/19.jpg)
Hybrid Approach
• Usability + Interac9vity • NL [4] + Graph [3] = Hybrid Query Approach • Builds on Advantages of either approach
– Quick access to relevant data – Easy and visual means to formulate complex queries – Visualise nearby concepts – Visually inspect search space
• Alleviates the disadvantages – Lesser 9me taken to build complex queries
[3] S.Mazumdar, D.Petrelli, K. Elbedweihy, V. Lanfranchi, and F. Ciravegna. Affec9ve Graphs: The visual appeal of linked data. Seman9c web : interoperability, usability, applicability, 2013. In Press. [4] K. Elbedweihy, S.N. Wrigley, F. Ciravegna, and Z. Zhang (2013). Using BabelNet in bridging the gap between natural language queries and linked data concepts. NLP & DBpedia Workshop, 12th Interna9onal Seman9c Web Conference (ISWC2013), 21-‐25 October, Sydney, Australia.
![Page 20: NL#Graphs:+A+Hybrid+Approach+toward+ …NL#Graphs:+A+Hybrid+Approach+toward+ Interac9vely+Querying+Seman9c+Data Khadija Elbedweihy+ Suvodeep+Mazumdar+ StuartN.+Wrigley+ Fabio+Ciravegna](https://reader035.vdocuments.mx/reader035/viewer/2022063001/5f1c6d137e60a701484d03d3/html5/thumbnails/20.jpg)
Quan9ta9ve Evalua9on
• Query Execu9on phase (98ms to 19.9s)
20
![Page 21: NL#Graphs:+A+Hybrid+Approach+toward+ …NL#Graphs:+A+Hybrid+Approach+toward+ Interac9vely+Querying+Seman9c+Data Khadija Elbedweihy+ Suvodeep+Mazumdar+ StuartN.+Wrigley+ Fabio+Ciravegna](https://reader035.vdocuments.mx/reader035/viewer/2022063001/5f1c6d137e60a701484d03d3/html5/thumbnails/21.jpg)
Evalua9ng.views. – Performance Test Results
21
Remote database Test Results – Variability between. 0.00026 ms to 6.48ms
![Page 22: NL#Graphs:+A+Hybrid+Approach+toward+ …NL#Graphs:+A+Hybrid+Approach+toward+ Interac9vely+Querying+Seman9c+Data Khadija Elbedweihy+ Suvodeep+Mazumdar+ StuartN.+Wrigley+ Fabio+Ciravegna](https://reader035.vdocuments.mx/reader035/viewer/2022063001/5f1c6d137e60a701484d03d3/html5/thumbnails/22.jpg)
10 Principles of Aesthe9cs for SW
• Review of literature 10 general guidelines &
22
Mazumdar, S., Petrelli, D., Elbedweihy, K., Lanfranchi, V., & Ciravegna, F. Affec9ve Graphs: The Visual Appeal of Linked Data. Seman9c Web: Interoperability, Usability, Applicability.
![Page 23: NL#Graphs:+A+Hybrid+Approach+toward+ …NL#Graphs:+A+Hybrid+Approach+toward+ Interac9vely+Querying+Seman9c+Data Khadija Elbedweihy+ Suvodeep+Mazumdar+ StuartN.+Wrigley+ Fabio+Ciravegna](https://reader035.vdocuments.mx/reader035/viewer/2022063001/5f1c6d137e60a701484d03d3/html5/thumbnails/23.jpg)
9 node-‐linked visualisa9on guidelines
![Page 24: NL#Graphs:+A+Hybrid+Approach+toward+ …NL#Graphs:+A+Hybrid+Approach+toward+ Interac9vely+Querying+Seman9c+Data Khadija Elbedweihy+ Suvodeep+Mazumdar+ StuartN.+Wrigley+ Fabio+Ciravegna](https://reader035.vdocuments.mx/reader035/viewer/2022063001/5f1c6d137e60a701484d03d3/html5/thumbnails/24.jpg)
Measuring Aesthe9cs
• Screenshots of interfaces fed-‐in to a Java program, where each element was “marked-‐up” using mouse gestures
24
D.C.L. Ngo, A. Samsudin, and R. Abdullah. Aesthe9c measures for assessing graphic screens. J. INF. SCI. ENG, 16(1):97–116, 2000. D.C.L. Ngo, L.S. Teo, and J.G. Byrne. Modelling interface aesthe9cs. Informa9on Sciences, 152:25–46, 2003.
![Page 25: NL#Graphs:+A+Hybrid+Approach+toward+ …NL#Graphs:+A+Hybrid+Approach+toward+ Interac9vely+Querying+Seman9c+Data Khadija Elbedweihy+ Suvodeep+Mazumdar+ StuartN.+Wrigley+ Fabio+Ciravegna](https://reader035.vdocuments.mx/reader035/viewer/2022063001/5f1c6d137e60a701484d03d3/html5/thumbnails/25.jpg)
Results
25
0.652
0.561