"Do you trust me?"

Weitz, Katharina; Schiller, Dominik; Schlagowski, Ruben; Huber, Tobias B.; André, Elisabeth

doi:10.1145/3308532.3329441

Cited by 86 publications

(4 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Transparency is known to be a significant factor in trusting automated data science systems. However, much of the literature states a lack of transparency in AI [5,64,65]. We believe exploring the information needs and individual differences of data scientists can inform us of possible ways to increase trust in AutoML tools.…”

Section: Resultsmentioning

confidence: 99%

Trust in AutoML

Drozdal

Weisz²,

Wang³

et al. 2020

Proceedings of the 25th International Conference on Intelligent User Interfaces

View full text Add to dashboard Cite

We explore trust in a relatively new area of data science: Automated Machine Learning (AutoML). In AutoML, AI methods are used to generate and optimize machine learning models by automatically engineering features, selecting models, and optimizing hyperparameters. In this paper, we seek to understand what kinds of information influence data scientists' trust in the models produced by AutoML? We operationalize trust as a willingness to deploy a model produced using automated methods. We report results from three studies -qualitative interviews, a controlled experiment, and a card-sorting task -to understand the information needs of data scientists for establishing trust in AutoML systems. We find that including transparency features in an AutoML tool increased user trust and understandability in the tool; and out of all proposed features, model performance metrics and visualizations are the most important information to data scientists when establishing their trust with an AutoML tool. CCS CONCEPTS• Human-centered computing → User studies; Empirical studies in HCI ; • Computing methodologies → Artificial intelligence.

show abstract

Section: Resultsmentioning

confidence: 99%

Trust in AutoML

Drozdal

Weisz²,

Wang³

et al. 2020

Proceedings of the 25th International Conference on Intelligent User Interfaces

View full text Add to dashboard Cite

show abstract

“…While existing methods discuss user trust as a broad goal, to the best of our knowledge, their ability to help regulators or decision-makers adequately trust ML models is not demonstrated through well-defined evaluations or user trials. The experimental work on the notion of trust has relied on subjective, self-reported measures of trust in performing a simplified task (Ribeiro et al, 2016; Weitz et al, 2019; Buçinca et al, 2020). However, Jacovi et al (2021) in their effort of formalizing the notion of trust in ML, argue that simply asking the user whether they trust the model for a simple task does not evaluate the notion of trust in AI, as the users are not assuming any risk, and they argue that relying on an AI with assumed risk is a prerequisite for trust.…”

Section: Current State Of Explainable MLmentioning

confidence: 99%

Explainable machine learning for public policy: Use cases, gaps, and research directions

et al. 2023

View full text Add to dashboard Cite

Explainability is highly desired in machine learning (ML) systems supporting high-stakes policy decisions in areas such as health, criminal justice, education, and employment. While the field of explainable ML has expanded in recent years, much of this work has not taken real-world needs into account. A majority of proposed methods are designed with generic explainability goals without well-defined use cases or intended end users and evaluated on simplified tasks, benchmark problems/datasets, or with proxy users (e.g., Amazon Mechanical Turk). We argue that these simplified evaluation settings do not capture the nuances and complexities of real-world applications. As a result, the applicability and effectiveness of this large body of theoretical and methodological work in real-world applications are unclear. In this work, we take steps toward addressing this gap for the domain of public policy. First, we identify the primary use cases of explainable ML within public policy problems. For each use case, we define the end users of explanations and the specific goals the explanations have to fulfill. Finally, we map existing work in explainable ML to these use cases, identify gaps in established capabilities, and propose research directions to fill those gaps to have a practical societal impact through ML. The contribution is (a) a methodology for explainable ML researchers to identify use cases and develop methods targeted at them and (b) using that methodology for the domain of public policy and giving an example for the researchers on developing explainable ML methods that result in real-world impact.

show abstract

“…"... systems that can explain their rationale to a human user, characterize their strengths and weaknesses, and convey an understanding of how they will behave in the future" [6] Anthropomorphising technology in this way has implications for technology acceptance (see [7]; and Section 4.3 below). Surveys by Adadi and Berrada [2] and Arrieta and his colleagues [5] focus primarily on mapping out the domain from recent research.…”

Section: Responsible and Explainable Aimentioning

confidence: 99%

“…The DARPA program seeks to support "explanation-informed acceptance" via an understanding of the socio-cognitive context of explanation [6]. Picking up on such a human-mediated approach, Weitz and her colleagues demonstrate how even simple methods, in their case the use of an avatar-like component, encourage and enhance perceptions of understanding the technology [7]. Taking this further and echoing [9] on trust, Israelsen and Ahmed, meanwhile, focus on trust-enhancing "algorithmic assurances" which echo traditional constructs like trustworthiness indicators in the trust literature (see Section 4.4) [11].…”

Section: Responsible and Explainable Aimentioning

confidence: 99%

Trust, but Verify: Informed Consent, AI Technologies, and Public Health Emergencies

Pickering

2021

Future Internet

View full text Add to dashboard Cite

To use technology or engage with research or medical treatment typically requires user consent: agreeing to terms of use with technology or services, or providing informed consent for research participation, for clinical trials and medical intervention, or as one legal basis for processing personal data. Introducing AI technologies, where explainability and trustworthiness are focus items for both government guidelines and responsible technologists, imposes additional challenges. Understanding enough of the technology to be able to make an informed decision, or consent, is essential but involves an acceptance of uncertain outcomes. Further, the contribution of AI-enabled technologies not least during the COVID-19 pandemic raises ethical concerns about the governance associated with their development and deployment. Using three typical scenarios—contact tracing, big data analytics and research during public emergencies—this paper explores a trust-based alternative to consent. Unlike existing consent-based mechanisms, this approach sees consent as a typical behavioural response to perceived contextual characteristics. Decisions to engage derive from the assumption that all relevant stakeholders including research participants will negotiate on an ongoing basis. Accepting dynamic negotiation between the main stakeholders as proposed here introduces a specifically socio–psychological perspective into the debate about human responses to artificial intelligence. This trust-based consent process leads to a set of recommendations for the ethical use of advanced technologies as well as for the ethical review of applied research projects.

show abstract

"Do you trust me?"

Cited by 86 publications

References 7 publications

Trust in AutoML

Trust in AutoML

Explainable machine learning for public policy: Use cases, gaps, and research directions

Trust, but Verify: Informed Consent, AI Technologies, and Public Health Emergencies

Contact Info

Product

Resources

About