Comparative Methodologies for Evaluation of Ontology Design

Silva-López, Rafaela Blanca; Méndez-Gurrola, Iris Iddaly; Leyva, Hugo Pablo

doi:10.1007/978-3-030-60887-3_9

Cited by 2 publications

(15 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, there are some subjective methods that can be used for evaluating both SKOS vocabularies and OWL ontologies. Silva-López et al [7] suggest a quantitative model of minimalist verification techniques (QMM) based on the ontology design principles, mentioned in Gruber [30], Köhler [31], and Wiesner and Marquardt [32]. Based on QMM, if the ontology is compliant with a principle, one point is assigned to it, which is adjustable based on how it fulfills the criteria.…”

Section: Skos/ontology Quality Evaluation Metrics and Approachesmentioning

confidence: 99%

“…As a continuation of this work, Suominen and Mader [8] define 26 quality issues and update qSKOS accordingly. In this study, QMM [7] and the 26 SKOS vocabulary quality evaluation metrics by Suominen and Mader [8] are used for analysis.…”

Section: Skos/ontology Quality Evaluation Metrics and Approachesmentioning

confidence: 99%

“…Second, a comprehensive set of metrics should be chosen to evaluate the quality of HIACS and to see if they can cover governance aspects. Accordingly, we use QMM [7], the SKOS vocabulary quality evaluation metrics by Suominen and Mader, and the qSKOS tool [8] to evaluate HAICS.…”

Section: Use Case and Requirementsmentioning

confidence: 99%

“…Using HAICS as our use case, we evaluate its FAIRness and mitigate its FAIRness limitations using FAIR principles best practices and recommendations for semantic artefacts [5], [6], [19], [20]. To stress the importance of assessing and mitigating governance issues in semantic artefacts, we also evaluate the quality of HAICS using 26 quality metrics for SKOS vocabularies [8] and the QMM subjective method [7] and map the QMM metrics to FAIR principles and FAccT. This allows checking if the metrics are able to point out FAIRness issues in semantic resources.…”

Section: Designmentioning

confidence: 99%

“…We also analyse if these recommendations and best practices support accountability and transparency. To analyse the quality of HAICS and the relation between ontology quality evaluation metrics, FAIR principles, and FAccT, we use one subjective method [7] and a set of quality evaluation metrics for SKOS vocabularies [8].…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

FAIR Ontologies for Transparent and Accountable AI: A Hospital Adverse Incidents Vocabulary Case Study

Basereh

Caputo

Brennan

2021

2021 Third International Conference on Transdisciplinary AI (TransAI)

View full text Add to dashboard Cite

In this paper, the relation between the FAIR (Findable, Accessible, Interoperable, Reusable) ontologies and accountability and transparency of ontology-based AI systems is analysed. Also, governance-related gaps in ontology quality evaluation metrics were identified by examining their relation with FAIR principles and FAcct (Fairness, Accountability, Transparency) governance aspects. A simple SKOS vocabulary, titled "Hospital Adverse Incidents Classification Scheme" (HAICS) has been used as a use case for this study. Theoretically, we found that there is a straight relation between FAIR principles and FAccT AI, which means that FAIR ontologies enhance transparency and accountability in ontology-based AI systems. We suggest that "FAIRness" should be assessed as one of the ontology quality evaluation aspects.

show abstract

Section: Skos/ontology Quality Evaluation Metrics and Approachesmentioning

confidence: 99%

Section: Skos/ontology Quality Evaluation Metrics and Approachesmentioning

confidence: 99%

Section: Use Case and Requirementsmentioning

confidence: 99%

Section: Designmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

FAIR Ontologies for Transparent and Accountable AI: A Hospital Adverse Incidents Vocabulary Case Study

Basereh

Caputo

Brennan

2021

2021 Third International Conference on Transdisciplinary AI (TransAI)

View full text Add to dashboard Cite

show abstract

AccTEF: A Transparency and Accountability Evaluation Framework for Ontology-Based Systems

Basereh

Caputo

Brennan

2022

Int. J. Semantic Computing

View full text Add to dashboard Cite

This paper proposes a new accountability and transparency evaluation framework (AccTEF) for ontology-based systems (OSysts). AccTEF is based on an analysis of the relation between a set of widely accepted data governance principles, i.e. findable, accessible, interoperable, reusable (FAIR) and accountability and transparency concepts. The evaluation of accountability and transparency of input ontologies and vocabularies of OSysts are addressed by analyzing the relation between vocabulary and ontology quality evaluation metrics, FAIR and accountability and transparency concepts. An ontology-based knowledge extraction pipeline is used as a use case in this study. Discovering the relation between FAIR and accountability and transparency helps in identifying and mitigating risks associated with deploying OSysts. This also allows providing design guidelines that help accountability and transparency to be embedded in OSysts. We found that FAIR can be used as a transparency indicator. We also found that the studied vocabulary and ontology quality evaluation metrics do not cover FAIR, accountability and transparency. Accordingly, we suggest these concepts should be considered as vocabulary and ontology quality evaluation aspects. To the best of our knowledge, it is the first time that the relation between FAIR and accountability and transparency concepts has been found and used for evaluation.

show abstract

Comparative Methodologies for Evaluation of Ontology Design

Cited by 2 publications

References 8 publications

FAIR Ontologies for Transparent and Accountable AI: A Hospital Adverse Incidents Vocabulary Case Study

FAIR Ontologies for Transparent and Accountable AI: A Hospital Adverse Incidents Vocabulary Case Study

AccTEF: A Transparency and Accountability Evaluation Framework for Ontology-Based Systems

Contact Info

Product

Resources

About