Abstract. Benchmarking approaches for ontology merging is challenging and has received little attention so far. A key problem is that there is in general no single best solution for a merge task and that merging may either be performed symmetrically or asymmetrically. As a first step to evaluate the quality of ontology merging solutions we propose the use of general metrics such as the relative coverage of the input ontologies, the compactness of the merge result as well as the degree of introduced redundancy. We use these metrics to evaluate three merge approaches for different merge scenarios.
MotivationOntologies and taxonomies are increasingly used to semantically categorize or annotate information, e.g., for e-commerce or e-science. For example, product catalogs of online shops or comparison portals categorize products to help users and applications finding relevant information. Since many ontologies refer to the same domain and to the same objects, there is a growing need to integrate or merge such related ontologies. The goal is to create a merged ontology providing a unified view on two or more input ontologies.Ontology merging is a challenging problem especially for large and heterogeneous ontologies and require semi-automatic approaches to reduce the manual effort. Several such merge approaches have already been proposed, however their relative quality is largely unknown. One increasingly adopted and promising idea is to decompose the complex integration problem into match and merge subtasks and leverage the advances made for automatic ontology and schema matching [13] to solve the first subproblem. The merge subtask can then utilize a match mapping identifying corresponding concepts in the input ontologies that should be merged. Such a match-based merging is followed in [11] [7]) are most common and aim at completely integrating all input ontologies with the same priority. Asymmetric approaches, by contrast, take one of the input ontologies as the target and merge the other input ontologies into this target [11] [15] [6] thereby giving preference to the target ontology.Given the different merge approaches we see an increasing need to quantitatively evaluate their quality and performance. For the subproblem of ontology matching such evaluations are now quite common [1] [3] and there is also a benchmark for determining schema mappings [2]. Typically the quality of a match algorithm is determined by evaluating it on some match problems for which a manually defined perfect match result is provided for comparison. While a similar approach for evaluating merge approaches has been advocated for in [4] we argue that there is in general no single perfect merge