AccTEF: A Transparency and Accountability Evaluation Framework for Ontology-Based Systems

Basereh, Maryam; Caputo, Annalina; Brennan, Rob

doi:10.1142/s1793351x22400013

Cited by 4 publications

(3 citation statements)

References 64 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Scientific assessment of the carbon footprint of tourism activities and the carbon absorption of natural ecosystems, i.e. the carbon balance of tourism ecosystems, is important for the quantitative analysis of the carbon balance of tourism activities and the formulation of low carbon management decisions in nature reserves [17][18].…”

Section: Carbon Revenue and Expenditure Evaluation Modelmentioning

confidence: 99%

Assessment System of Natural Environment Protection Based on Energy Efficiency

2023

NEP

View full text Add to dashboard Cite

Since the reform and opening up, China's economy has developed at a rapid pace, and the impact of industrial development and social change on the natural environment has become more diverse and profound. In evaluating the economic value of nature reserves, we often have to consider their current economic value, and past values appear to be meaningless, which makes it necessary to ensure both a reliable basis for the calculation criteria when calculating the value of reserves, and the use of time-recent This makes it necessary to ensure that the criteria for calculating the value of protected areas are reliable and that the most recent data are used. Therefore, this paper uses Nature Reserve C as an example to construct an assessment system for natural environment protection based on energy efficiency. The paper begins with a brief introduction of the evaluation principles and the carbon balance evaluation model, followed by the design of the nature reserve evaluation system and the general architecture of the system, and finally the analysis of the natural resource value and the economic value of the nature reserve.

show abstract

Section: Carbon Revenue and Expenditure Evaluation Modelmentioning

confidence: 99%

Assessment System of Natural Environment Protection Based on Energy Efficiency

2023

NEP

View full text Add to dashboard Cite

show abstract

“…This criteria has been adopted from three transparency checklists [50,51,52] which include reproducibility and transparency indicators of scientific papers and reports. An adaptation of Jalali et al's transparency assessment checklist for the system transparency evaluation was used in our previous work [53].…”

Section: Existing Solutions For the Evaluation Of Ai Systems' Transpa...mentioning

confidence: 99%

Automatic transparency evaluation for open knowledge extraction systems

Basereh

Caputo

Brennan

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Background: In this paper a new transparency evaluation framework, Cyrus, is proposed for Open Knowledge Extraction (OKE) systems. Cyrus is based on the state-of-the-art AI and data transparency methods and a well-accepted framework for linked data quality evaluation. Then, the transparency of the outputs of three state-of-the-art OKE systems are automatically evaluated using a combination of three state-of-the-art FAIR (Findable, Accessible, Interoperable, Reusable) tools and the Luzzu linked data quality evaluation framework. OKE systems extract structured knowledge from unstructured or semi-structured text and publish that as linked data. These systems can be used as the fundamental component of advanced knowledge services. However, due to the transition from rule-based to black-box machine learning approaches, most OKE systems are not transparent. This means that their processes and outcomes are not understandable and interpretable for humans. As a result, the transparency problem has been identified as critical by European Union Trustworthy AI guidelines. Automatic transparency evaluation helps with scalability and gives a transparency score that allows comparing different systems' transparencies. It also gives insight into the transparency weaknesses of the system and how to enhance the transparency of the system. Accordingly, being able to evaluate the transparency can be the first step of an effective transparency enhancement process. Results: In Cyrus, data transparency includes 27 dimensions which are grouped in two categories. In this paper, Six of these dimensions, i.e., provenance, interpretability, understandability, licensing, availability, interlinking have been evaluated automatically for three state-of-the-art OKE systems, using the state-of-the-art metrics and tools. The highest mean transparency has been approx. 31\%. Conclusions: This is the first research to study automated transparency evaluation for OKE systems. We show that state-of-the-art FAIR assessment tools and linked data quality evaluation techniques are capable of identifying some transparency weaknesses of OKE systems.\end{abstract}

show abstract

“…These criteria have been adopted from three transparency checklists [ 50 – 52 ] which include reproducibility and transparency indicators for scientific papers and reports. Jalali et al’s transparency assessment checklist was used in [ 53 ] for the transparency evaluation.…”

Section: Introductionmentioning

confidence: 99%

Automatic transparency evaluation for open knowledge extraction systems

Basereh,

Caputo,

Brennan

2023

J Biomed Semant

Self Cite

View full text Add to dashboard Cite

Background This paper proposes Cyrus, a new transparency evaluation framework, for Open Knowledge Extraction (OKE) systems. Cyrus is based on the state-of-the-art transparency models and linked data quality assessment dimensions. It brings together a comprehensive view of transparency dimensions for OKE systems. The Cyrus framework is used to evaluate the transparency of three linked datasets, which are built from the same corpus by three state-of-the-art OKE systems. The evaluation is automatically performed using a combination of three state-of-the-art FAIRness (Findability, Accessibility, Interoperability, Reusability) assessment tools and a linked data quality evaluation framework, called Luzzu. This evaluation includes six Cyrus data transparency dimensions for which existing assessment tools could be identified. OKE systems extract structured knowledge from unstructured or semi-structured text in the form of linked data. These systems are fundamental components of advanced knowledge services. However, due to the lack of a transparency framework for OKE, most OKE systems are not transparent. This means that their processes and outcomes are not understandable and interpretable. A comprehensive framework sheds light on different aspects of transparency, allows comparison between the transparency of different systems by supporting the development of transparency scores, gives insight into the transparency weaknesses of the system, and ways to improve them. Automatic transparency evaluation helps with scalability and facilitates transparency assessment. The transparency problem has been identified as critical by the European Union Trustworthy Artificial Intelligence (AI) guidelines. In this paper, Cyrus provides the first comprehensive view of transparency dimensions for OKE systems by merging the perspectives of the FAccT (Fairness, Accountability, and Transparency), FAIR, and linked data quality research communities. Results In Cyrus, data transparency includes ten dimensions which are grouped in two categories. In this paper, six of these dimensions, i.e., provenance, interpretability, understandability, licensing, availability, interlinking have been evaluated automatically for three state-of-the-art OKE systems, using the state-of-the-art metrics and tools. Covid-on-the-Web is identified to have the highest mean transparency. Conclusions This is the first research to study the transparency of OKE systems that provides a comprehensive set of transparency dimensions spanning ethics, trustworthy AI, and data quality approaches to transparency. It also demonstrates how to perform automated transparency evaluation that combines existing FAIRness and linked data quality assessment tools for the first time. We show that state-of-the-art OKE systems vary in the transparency of the linked data generated and that these differences can be automatically quantified leading to potential applications in trustworthy AI, compliance, data protection, data governance, and future OKE system design and testing.

show abstract

AccTEF: A Transparency and Accountability Evaluation Framework for Ontology-Based Systems

Cited by 4 publications

References 64 publications

Assessment System of Natural Environment Protection Based on Energy Efficiency

Assessment System of Natural Environment Protection Based on Energy Efficiency

Automatic transparency evaluation for open knowledge extraction systems

Automatic transparency evaluation for open knowledge extraction systems

Contact Info

Product

Resources

About