1. Protg4US: Harvesting Ontology Authoring Data with Protg Markel Vigo, Caroline Jay, Robert Stevens firstname.lastname@manchester.ac.uk @markelvigo, @CarolineEJay, @stevensrd65 Workshop on Human-Semantic Web Interaction, HSWI 2014 May 26. Crete (Greece)

2. Little is known about the human factors of ontology authoring Protg4US: Harvesting Ontology Authoring Data with Protg HSWI @ ESWC 2014 A lot of what we know is based on anecdotal evidence We asked about problems and strategies Making sense Search and retrieval Efficient population On-the-fly reasoning Overloaded explanations Lack of evaluation methods Design insights for the next wave ontology authoring tools. CHI 2014 http://dx.doi.org/10.1145/2556288.2557284 Introduction > Protg4US > Results > Conclusion 3. Lack of sound HCI research in the Semantic Web Protg4US: Harvesting Ontology Authoring Data with Protg HSWI @ ESWC 2014 HCI approaches do not pervade all computing disciplines Instruments to run user studies are scarce Consequences for the OWL realm No real understanding about the authoring process Authoring tools are not human-centered What if we want to go further? Automatic detection of authoring patterns Intelligent support for authoring Introduction > Protg4US > Results > Conclusion 4. Protg4US: a step towards having observational instruments Protg4US: Harvesting Ontology Authoring Data with Protg HSWI @ ESWC 2014 Protg4US: Protg for User Studies Logging capabilities of: Interaction events: click, hover, expand hierarchy... Authoring events: add siblings, add restrictions... Environment commands: reason, search, undo... 76585,2,Classes,Element edited,Juliette subclass of: Potato and hasCroppingTime some Main cropping 77786,3,Classes,Save ontology,http://owl.cs.manchester.ac.uk/ontology/start-here.owl 80204,3,Classes,Reasoner invoked,HermiT 1.3.8 80647,1,Classes,Mouse entered, Class hierarchy (inferred) 82910,1,Classes,Element hovered,Early_cropping_potato 83049,1,Classes,Element selected,Early_cropping_potato 83661,1,Classes,Hierarchy expanded,Early_cropping_potato Introduction > Protg4US > Results > Conclusion 5. User study to show the strengths of Protg4US Protg4US: Harvesting Ontology Authoring Data with Protg HSWI @ ESWC 2014 Experimental design: Participants: 15 expert authors Stimuli: a potato ontology and Protg4US 3 authoring tasks with an increased complexity Collected data (apart from Protg4US logs): Completion times Self reported expertise Perceived task difficulty Screen video and eye-tracking Introduction > Protg4US > Results > Conclusion 6. Protg4US in action Protg4US: Harvesting Ontology Authoring Data with Protg HSWI @ ESWC 2014 Introduction > Protg4US > Results > Conclusion 7. Data analysis to check the intention of the experimental design Protg4US: Harvesting Ontology Authoring Data with Protg HSWI @ ESWC 2014 Tasks had an increased complexity Completion times significantly different: T1 Protg4US > Results > Conclusion 8. Some events may be indicators of problematic situations Protg4US: Harvesting Ontology Authoring Data with Protg HSWI @ ESWC 2014 Positive and significant correlations between task completion time and: Expansions of the class hierarchy Running the reasoner Renaming entities Log analysis allows to profile users based on their tab use Introduction > Protg4US > Results > Conclusion 9. Reconstructing the interaction allows to identify patterns through visualisation Protg4US: Harvesting Ontology Authoring Data with Protg HSWI @ ESWC 2014 Web diagrams show the most frequent transitions between statesP15 log Back Check property Class addition Convert into defined class:finished Convert into defined class:start Entity renamed Explanation invoked Entity deleted Entity dragged Entity edited:finishEntity edited:start Entity selected Link clicked Property addition Reasoner finished Reasoner invoked Save Set active ontology Hierarchy collapsed Hierarchy expanded Undo P9 log Back Check property Class addition Convert into defined class:finished Convert into defined class:start Entity renamed Explanation invoked Entity deleted Entity dragged Entity edited:finishEntity edited:start Entity selected Link clicked Property addition Reasoner finished Reasoner invoked Save Set active ontology Hierarchy collapsed Hierarchy expanded Undo Introduction > Protg4US > Results > Conclusion 10. Reconstructing the interaction allows to identify patterns through visualisation Protg4US: Harvesting Ontology Authoring Data with Protg HSWI @ ESWC 2014 Time diagrams show the authoring rhythm Introduction > Protg4US > Results > Conclusion 11. Reconstructing the interaction allows to identify patterns through visualisation Protg4US: Harvesting Ontology Authoring Data with Protg HSWI @ ESWC 2014 The analysis of diagrams across users allows to sketch decision trees Reasoner invoked after 1. A class is converted into a defined class 2. Ontology is saved 3. An entity is selected Hierarchy is expanded after 1. Reasoner finishes 2. Hierarchy is expanded 3. Entity addition is invoked Introduction > Protg4US > Results > Conclusion 12. Future work and Conclusions Protg4US: Harvesting Ontology Authoring Data with Protg HSWI @ ESWC 2014 Future work Statistical analysis of patterns and identification of strategies Incorporation of eye-tracking data Ontology authoring into the wild Conclusions Protg4US may give urgency to a more human- centred Semantic Web Better tools for ontology and linked data authoring Introduction > Protg4US > Results > Conclusion 13. Protg4US: Harvesting Ontology Authoring Data with Protg Markel Vigo, Caroline Jay, Robert Stevens firstname.lastname@manchester.ac.uk @markelvigo, @CarolineEJay, @stevensrd65 Workshop on Human-Semantic Web Interaction, HSWI 2014 May 26. Crete (Greece) WhatIf: Answering What if... questions for Ontology Authoring. EPSRC reference EP/J014176/1