Intelligence brings responsibility - Even smart AI assistants are held responsible

Longin, Louis; Bahrami, Bahador; Deroy, Ophelia

doi:10.1016/j.isci.2023.107494

Cited by 8 publications

(2 citation statements)

References 62 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Note, however, that until AI can be designed to exhibit an awareness of right and wrong, it would be a conceptual mistake to transfer our praise or blame onto an artificial agent. The argument here is not that our practices of blame and praise are biased or flawed (see, e.g., Longin et al, 2023 andPorsdam et al, 2023, for empirical and normative discussion of this point). The argument is that our practices demand that the goal of moral evaluation is to shape the recipient's conscience, structure her cognitive repertoire to better align with shared values.…”

Section: Why Morally Trustworthy Ai Is Unnecessarymentioning

confidence: 98%

“Quasi-Metacognitive Machines: Why We Don’t Need Morally Trustworthy AI and Communicating Reliability is Enough”

Dorsch,

Deroy

2024

Philos. Technol.

Self Cite

View full text Add to dashboard Cite

Many policies and ethical guidelines recommend developing “trustworthy AI”. We argue that developing morally trustworthy AI is not only unethical, as it promotes trust in an entity that cannot be trustworthy, but it is also unnecessary for optimal calibration. Instead, we show that reliability, exclusive of moral trust, entails the appropriate normative constraints that enable optimal calibration and mitigate the vulnerability that arises in high-stakes hybrid decision-making environments, without also demanding, as moral trust would, the anthropomorphization of AI and thus epistemically dubious behavior. The normative demands of reliability for inter-agential action are argued to be met by an analogue to procedural metacognitive competence (i.e., the ability to evaluate the quality of one’s own informational states to regulate subsequent action). Drawing on recent empirical findings that suggest providing reliability scores (e.g., F1-scores) to human decision-makers improves calibration in the AI system, we argue that reliability scores provide a good index of competence and enable humans to determine how much they wish to rely on the system.

show abstract

Section: Why Morally Trustworthy Ai Is Unnecessarymentioning

confidence: 98%

“Quasi-Metacognitive Machines: Why We Don’t Need Morally Trustworthy AI and Communicating Reliability is Enough”

Dorsch,

Deroy

2024

Philos. Technol.

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, the increasing development of AI systems that assist and collaborate with humans, rather than replacing them (Balazadeh Meresht et al, 2022;De et al, 2020De et al, , 2021Mozannar et al, 2022;Okati et al, 2021;Raghu et al, 2019;Straitouri et al, 2021;Wilder et al, 2021), calls for more empirical and theoretical research to shed light on the way humans make responsibility judgments in situations involving human-AI teams (Cañas, 2022). Recent work in that area has identified several factors that influence responsibility judgments (Awad et al, 2020;Lima et al, 2021;Longin et al, 2023). However, this work has not attempted to characterize the underlying cognitive processes that support such judgments.…”

Section: Introductionmentioning

confidence: 99%

Towards a computational model of responsibility judgments in sequential human-AI collaboration

Tsirtsis,

Rodriguez,

Gerstenberg

2024

Preprint

View full text Add to dashboard Cite

When a human and an AI agent collaborate to complete a task and something goes wrong, who is responsible? Prior work has developed theories to describe how people assign responsibility to individuals in teams. However, there has been little work studying the cognitive processes that underlie responsibility judgments in human-AI collaborations, especially for tasks comprising a sequence of interdependent actions. In this work, we take a step towards filling this gap. Using semi-autonomous driving as a paradigm, we develop an environment that simulates stylized cases of human-AI collaboration using a generative model of agent behavior. We propose a model of responsibility that considers how unexpected an agent’s action was, and what would have happened had they acted differently. We test the model’s predictions empirically and find that in addition to action expectations and counterfactual considerations, participants’ responsibility judgments are also affected by how much each agent actually contributed to the outcome.

show abstract

Perceived responsibility in AI-supported medicine

Krügel,

Ammeling,

Aubreville

et al. 2024

AI & Soc

View full text Add to dashboard Cite

In a representative vignette study in Germany with 1,653 respondents, we investigated laypeople’s attribution of moral responsibility in collaborative medical diagnosis. Specifically, we compare people’s judgments in a setting in which physicians are supported by an AI-based recommender system to a setting in which they are supported by a human colleague. It turns out that people tend to attribute moral responsibility to the artificial agent, although this is traditionally considered a category mistake in normative ethics. This tendency is stronger when people believe that AI may become conscious at some point. In consequence, less responsibility is attributed to human agents in settings with hybrid diagnostic teams than in settings with human-only diagnostic teams. Our findings may have implications for behavior exhibited in contexts of collaborative medical decision making with AI-based as opposed to human recommenders because less responsibility is attributed to agents who have the mental capacity to care about outcomes.

show abstract

Intelligence brings responsibility - Even smart AI assistants are held responsible

Cited by 8 publications

References 62 publications

“Quasi-Metacognitive Machines: Why We Don’t Need Morally Trustworthy AI and Communicating Reliability is Enough”

“Quasi-Metacognitive Machines: Why We Don’t Need Morally Trustworthy AI and Communicating Reliability is Enough”

Towards a computational model of responsibility judgments in sequential human-AI collaboration

Perceived responsibility in AI-supported medicine

Contact Info

Product

Resources

About