What Information is Required for Explainable AI? : A Provenance-based Research Agenda and Future Challenges

Jaigirdar, Fariha Tasmin; Rudolph, Carsten; Oliver, Gillian; Watts, David; Bain, Chris

doi:10.1109/cic50333.2020.00030

Cited by 8 publications

(6 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It defines concepts and relationships to capture entities involved in a process, activities that took place, and their interconnections. The 'Six-Ws' framework proposed by Jaigirdar et al [51] provides another idea for the design of XAI provenance.…”

Section: Xai Data Provenancementioning

confidence: 99%

An Open API Architecture to Discover the Trustworthy Explanation of Cloud AI Services

Wang,

Liu,

Huang

2023

Preprint

View full text Add to dashboard Cite

show abstract

Section: Xai Data Provenancementioning

confidence: 99%

An Open API Architecture to Discover the Trustworthy Explanation of Cloud AI Services

Wang,

Liu,

Huang

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…To increase understanding, outputs should be visualised where possible, making them easier for users to interpret [169]. To assist in transparency, users should be provided with sufficient meta-information about the model (including addressing the questions 'who', 'which', 'what', 'why', 'when' and 'where'), an approach further detailed in the explainable AI framework provided by [146].…”

Section: Stakeholder Engagement and User-centered Designmentioning

confidence: 99%

“…At present, a key barrier and opportunity is that AI developers have only limited awareness of existing solutions that support ethical AI development, with organisational barriers persisting [186]. For example, developers may be unaware how to document their datasets [102] or to present their models in an explainable manner [146], despite these guidelines being available. As suggested by [186], these issues could be overcome by simple organisational mechanisms, rather than technical solutions.…”

Section: Improving Organisational Engagement Amongst Ai Developersmentioning

confidence: 99%

Operationalising ethics in artificial intelligence for healthcare: a framework for AI developers

2022

View full text Add to dashboard Cite

Artificial intelligence (AI) offers much promise for improving healthcare. However, it runs the looming risk of causing individual and societal harms; for instance, exacerbating inequalities amongst minority groups, or enabling compromises in the confidentiality of patients’ sensitive data. As such, there is an expanding, unmet need for ensuring AI for healthcare is developed in concordance with human values and ethics. Augmenting “principle-based” guidance that highlight adherence to ethical ideals (without necessarily offering translation into actionable practices), we offer a solution-based framework for operationalising ethics in AI for healthcare. Our framework is built from a scoping review of existing solutions of ethical AI guidelines, frameworks and technical solutions to address human values such as self-direction in healthcare. Our view spans the entire length of the AI lifecycle: data management, model development, deployment and monitoring. Our focus in this paper is to collate actionable solutions (whether technical or non-technical in nature), which can be steps that enable and empower developers in their daily practice to ensuring ethical practices in the broader picture. Our framework is intended to be adopted by AI developers, with recommendations that are accessible and driven by the existing literature. We endorse the recognised need for ‘ethical AI checklists’ co-designed with health AI practitioners, which could further operationalise the technical solutions we have collated. Since the risks to health and wellbeing are so large, we believe a proactive approach is necessary for ensuring human values and ethics are appropriately respected in AI for healthcare.

show abstract

“…Moreover, as soon as attacks can have serious consequences to human life or create significant financial damage, it becomes a major concern that end-users are not able to perceive any potential risks or attacks [11], [12] and end-users are not able to estimate or interpret if the data they see is trustworthy [10]. Thus, users do not have suitable cyber-situational awareness [13], [14] to know whether cyber-attacks are possible or have even occurred while the data was propagating through IoT systems.…”

Section: Introductionmentioning

confidence: 99%

Security-Aware Provenance for Transparency in IoT Data Propagation

et al. 2023

Self Cite

View full text Add to dashboard Cite

A successful application of an Internet of Things (IoT) based network depends on the accurate and successful delivery of data collected from numerous sources. A significant concern in IoT systems arises when end-users do not have sufficient transparency and are unaware of any potential data manipulation and risk in each step involved in data propagation. One potential solution is to integrate security metadata in IoT-based security-aware provenance graphs that provides better transparency with security awareness at each step of data propagation. In this paper, we integrate security metadata into the provenance graph with predefined security policies. We design a hypothetical IoT-Health scenario with possible threats: node cloning, fault packet injection, denial of service, unauthorized access, and malicious code injection. We simulate these threats in six cases to identify relevant risks. Our findings show how a security-aware provenance graph can offer end users greater transparency and security awareness by identifying failed signature verification (case 1), denial of service (case 2), unauthorized access (case 3), intrusion detection (case 4), missing WAF (case 5), and permission violation (case 6). We evaluate the transparency through obtaining authentication, integrity, availability and detecting underlying threats. Accordingly, this study promotes better risk assessment and decision-making for users with negligible performance overhead.INDEX TERMS Internet of Things (IoT), data provenance, IoT-Health, transparency, security-awareness.

show abstract

What Information is Required for Explainable AI? : A Provenance-based Research Agenda and Future Challenges

Cited by 8 publications

References 17 publications

An Open API Architecture to Discover the Trustworthy Explanation of Cloud AI Services

An Open API Architecture to Discover the Trustworthy Explanation of Cloud AI Services

Operationalising ethics in artificial intelligence for healthcare: a framework for AI developers

Security-Aware Provenance for Transparency in IoT Data Propagation

Contact Info

Product

Resources

About