Analysis of User Editing Patterns in Ontology Development Projects

Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems

Stevens

2015

Ontologies are complex knowledge representation artefacts used widely across biomedical, media and industrial domains. They are used for defining terminologies and providing metadata, especially for linked open data, and as such their use is rapidly increasing, but so far development tools have not benefited from empirical research into the ontology authoring process. This paper presents the results of a study that identifies common activity patterns through analysis of eyetracking data and the event logs of the popular authoring tool, Protégé. Informed by the activity patterns discovered, we propose design guidelines for bulk editing, efficient reasoning and increased situational awareness. Methodological implications go beyond the remit of knowledge artefacts: we establish a method for studying the usability of software designed for highly specialised complex domains.

Section: Introductionmentioning

confidence: 99%

Constructing Conceptual Knowledge Artefacts

Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems

Stevens

2015

“…For a detailed discussion of ontology authoring activities see [14]. Research on ontology authoring has experienced a resurgence in recent years [12,14,15]. One reason for this might be the increased utilisation of change-logs for ontology development.…”

Section: Background and Related Workmentioning

confidence: 99%

“…One reason for this might be the increased utilisation of change-logs for ontology development. WebProtégé for example produces change-logs during ontology authoring, which can form the basis of rich and informative analyses on ontology authoring activities [15]. While ontology authoring is increasingly performed in a programmatic fashion, a large number of ontologies have been built using ontology authoring environments such as Protégé [5] and WebProtégé [11].…”

Section: Background and Related Workmentioning

confidence: 99%

Making Entailment Set Changes Explicit Improves the Understanding of Consequences of Ontology Authoring Actions

Matentzoglu

Lecture Notes in Computer Science

et al. 2016

Abstract. The consequences of adding or removing axioms are difficult to apprehend for ontology authors using the Web Ontology Language (OWL). Consequences of modelling actions range from unintended inferences to outright defects such as incoherency or even inconsistency. One of the central ontology authoring activities is verifying that a particular modelling step has had the intended consequences, often with the help of reasoners. For users of Protégé, this involves, for example, exploring the inferred class hierarchy. We explore the hypothesis that making changes to key entailment sets explicit improves verification compared to the standard static hierarchy/framebased approach. We implement our approach as a Protégé plugin and conduct an exploratory study to isolate the authoring actions for which users benefit from our approach. In a second controlled study we address our hypothesis and find that, for a set of key authoring problems, making entailment set changes explicit improves the understanding of consequences both in terms of correctness and speed, and is rated as the preferred way to track changes compared to a static hierarchy/framebased view.

“…We argue that selfreports should preferably be complemented with objective interaction data. In this sense, the advent of web based authoring environments such as WebProtégé [5] will bring about not only the remote involvement of users, but will also facilitate the analysis of server log data as shown in [7]. While server log analysis allows to increase the involvement of geographically distributed participants, it has several downsides including the lack of contextual information or the difficulty in identifying the tasks and goals of users.…”

Section: Related Workmentioning

confidence: 99%

Protégé4US: Harvesting Ontology Authoring Data with Protégé

Lecture Notes in Computer Science

Stevens

2014

Abstract. The inherent complexity of ontologies poses a number of cognitive and perceptual challenges for ontology authors. We investigate how users deal with the complexity of the authoring process by analysing how one of the most widespread ontology development tools (i.e. Protégé) is used. To do so, we build Protégé4US (Protégé for User Studies) by extending Protégé in order to generate log files that contain ontology authoring events. These log files not only contain data about the interaction with the environment, but also about OWL entities and axioms. We illustrate the usefulness of Protégé4US with a case study with 15 participants. The data generated from the study allows us to know more about how Protégé is used (e.g. most frequently used tabs), how well users perform (e.g. task completion times) and identify emergent authoring strategies, including moving down the class hierarchy or saving the current workspace before running the reasoner. We argue that Protégé4US is an valuable instrument to identify ontology authoring patterns.