User evaluation of ontology as query construction tool

Abstract

This study examines the use of an ontology as a search tool. Sixteen subjects created queries using Concept-based Information Retrieval Interface (CIRI) and a regular baseline IR interface. The simulated work task method was used to make the searching situations realistic. Subjects' search experiences, queries and search results were examined. The numbers of search concepts and keys, as well as their overlap in the queries were investigated. The effectiveness of the CIRI and baseline queries was compared. An Ontology Index (OI) was calculated for all search tasks and the correlation between the OI and the overlap of search concepts and keys in queries was investigated. The number of search keys and concepts was higher in CIRI queries than in baseline interface queries. Also the overlap of search keys was higher among CIRI users than among baseline users. These both findings are due to CIRI's expansion feature. There was no clear correlation between OI and overlap of search concepts and keys. The search results were evaluated with generalised precision and recall, and relevance scores based on individual relevance assessments. The baseline interface queries performed better in all comparisons, but the difference was statistically significant only in relevance scores based on individual relevance assessments.

Informetric Analyses of Knowledge Organization Systems (KOSs)

“…They were asked to express their opinions on a scale between 0 and 5 (first questionnaire) and 1 and 10 (second questionnaire). Suomela and Kekäläinen (2006) evaluated an ontology as a query construction tool.…”

“…Wang, Khoo and Chaudhry (2014) evaluated the navigational effectiveness of a classification system. Both studies worked with experiments (task-based test method) as well as with interviews (Wang, Khoo, & Chaudhry, 2014) or questionnaires (Suomela & Kekäläinen, 2006).…”

A framework for evaluating automatic indexing or classification in the context of retrieval

“…Study 1 used a fact‐finding task and a research‐based searching task, and Study 2 a simple and complex searching task. Suomela and Kekäläinen () used a general simulated work task, giving rise to four detailed search tasks. The work task was writing a fairly large report on a given theme.…”

Ontology‐based Multi‐agent System to Support Business Users and Management / Daugiaagentine Sistema Grindžiama Ontologija Verslo Vartotojams Ir Vadybininkams Paremti

“…Furthermore research has extended to Web DW (Marotta, et al 2002) with the emphasis on managing the volatile and dynamic nature of Web sources. Utilization of ontologies is also addressed in Information Retrieval (IR) where it has been used for fuzzy tagging of data from the Web (Buche, et al 2006, Macias andCastells 2007), query construction tool in semi-automatic ontology mapping (Suomela and Kekalainen 2006) and semantic based retrieval of information from the World Wide Web (Shan et al 2003;Garces et al 2006). Use of ontologies in Data Mining (DM) has also been considered in (Bernaras et al 1996;Zhou et al 2002;Singh et al 2003;Cao et al 2004) where ontology was used for representation of context awareness and handling semantics inconsistencies.…”