Trust and Artificial Intelligence

Stanton, Brian; Jensen, Theodore

doi:10.6028/nist.ir.8332-draft

Cited by 41 publications

(27 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Trustworthy AI is a governance framework designed to mitigate potential adverse impacts on consumers as AI is poised to profoundly and indelibly change our lives. As mentioned in [17], Trustworthy AI is changing the dynamic between user and system into a relationship.…”

Section: Trustworthy Aimentioning

confidence: 99%

Trustworthy Artificial Intelligence and Process Mining: Challenges and Opportunities

Pery¹,

Rafiei²,

Simon³

et al. 2021

Preprint

View full text Add to dashboard Cite

The premise of this paper is that compliance with Trustworthy AI governance best practices and regulatory frameworks is an inherently fragmented process spanning across diverse organizational units, external stakeholders, and systems of record, resulting in process uncertainties and in compliance gaps that may expose organizations to reputational and regulatory risks. Moreover, there are complexities associated with meeting the specific dimensions of Trustworthy AI best practices such as data governance, conformance testing, quality assurance of AI model behaviors, transparency, accountability, and confidentiality requirements. These processes involve multiple steps, hand-offs, re-works, and human-in-the-loop oversight. In this paper, we demonstrate that process mining can provide a useful framework for gaining fact-based visibility to AI compliance process execution, surfacing compliance bottlenecks, and providing for an automated approach to analyze, remediate and monitor uncertainty in AI regulatory compliance processes.

show abstract

Section: Trustworthy Aimentioning

confidence: 99%

Trustworthy Artificial Intelligence and Process Mining: Challenges and Opportunities

Pery¹,

Rafiei²,

Simon³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Explainable AI is one of several properties that characterize trust in AI systems [121,127,134]. Other properties include resiliency, reliability, bias, and accountability.…”

Section: Introductionmentioning

confidence: 99%

Four principles of explainable artificial intelligence

Phillips¹,

Hahn²,

Fontana³

et al. 2021

104

View full text Add to dashboard Cite

We introduce four principles for explainable artificial intelligence (AI) that comprise fundamental properties for explainable AI systems. We propose that explainable AI systems deliver accompanying evidence or reasons for outcomes and processes; provide explanations that are understandable to individual users; provide explanations that correctly reflect the system's process for generating the output; and that a system only operates under conditions for which it was designed and when it reaches sufficient confidence in its output. We have termed these four principles as explanation, meaningful, explanation accuracy, and knowledge limits, respectively. Through significant stakeholder engagement, these four principles were developed to encompass the multidisciplinary nature of explainable AI, including the fields of computer science, engineering, and psychology. Because one-sizefits-all explanations do not exist, different users will require different types of explanations. We present five categories of explanation and summarize theories of explainable AI. We give an overview of the algorithms in the field that cover the major classes of explainable algorithms. As a baseline comparison, we assess how well explanations provided by people follow our four principles. This assessment provides insights to the challenges of designing explainable AI systems.

show abstract

“…Interpretability technology, including explainability, transparency, understandability, legibility, and intelligibility -generally considered as the ability to understand the internal logic, inner workings, and rationale behind predictions -are widely touted as a critical and necessary tool for trust. NIST, a U.S. national lab that aims to influence technology standards, writes that explainability is necessary to determine that an AI system is trustworthy [71]. An editorial in Nature Biomedical Engineering writes "for trust... opening up algorithms to interpretation is a necessary first step" [2].…”

Section: Introductionmentioning

confidence: 99%

Trust in AI: Interpretability is not necessary or sufficient, while black-box interaction is necessary and sufficient

Shen¹

2022

Preprint

View full text Add to dashboard Cite

The problem of human trust in artificial intelligence is one of the most fundamental problems in applied machine learning. Our processes for evaluating AI trustworthiness have substantial ramifications for ML's impact on science, health, and humanity, yet confusion surrounds foundational concepts. What does it mean to trust an AI, and how do humans assess AI trustworthiness? What are the mechanisms for building trustworthy AI? And what is the role of interpretable ML in trust?Here, we draw from statistical learning theory and sociological lenses on human-automation trust to motivate an AI-as-tool framework, which distinguishes human-AI trust from human-AI-human trust. Evaluating an AI's contractual trustworthiness involves predicting future model behavior using behavior certificates (BCs) that aggregate behavioral evidence from diverse sources including empirical out-of-distribution and out-of-task evaluation and theoretical proofs linking model architecture to behavior.We clarify the role of interpretability in trust with a ladder of model access. Interpretability (level 3) is not necessary or even sufficient for trust, while the ability to run a black-box model at-will (level 2) is necessary and sufficient. While interpretability can offer benefits for trust, it can also incur costs. We clarify ways interpretability can contribute to trust, while questioning the perceived centrality of interpretability to trust in popular discourse.How can we empower people with tools to evaluate trust? Instead of trying to understand how a model works, we argue for understanding how a model behaves. Instead of opening up black boxes, we should create more behavior certificates that are more correct, relevant, and understandable. We discuss how to build trusted and trustworthy AI responsibly with contract-aware model design, robustness testing, and favoring trust calibration over maximization. CCS Concepts: • Computing methodologies → Artificial intelligence; • Social and professional topics → Computing / technology policy; • Human-centered computing → Interaction design theory, concepts and paradigms.

show abstract

Trust and Artificial Intelligence

Cited by 41 publications

References 20 publications

Trustworthy Artificial Intelligence and Process Mining: Challenges and Opportunities

Trustworthy Artificial Intelligence and Process Mining: Challenges and Opportunities

Four principles of explainable artificial intelligence

Trust in AI: Interpretability is not necessary or sufficient, while black-box interaction is necessary and sufficient

Contact Info

Product

Resources

About