Deference to opaque systems and morally exemplary decisions

Abstract

Many have recently argued that there are weighty reasons against making high-stakes decisions solely on the basis of recommendations from artificially intelligent (AI) systems. Even if deference to a given AI system were known to reliably result in the right action being taken, the argument goes, that deference would lack morally important characteristics: the resulting decisions would not, for instance, be based on an appreciation of right-making reasons. Nor would they be performed from moral virtue; nor would they have moral worth. I argue that, even if these characteristics all have intrinsic value, that intrinsic value has no practical relevance to decisions about whether to defer to AI. I make that point by drawing on a lesson from the literature on moral testimony. Once it is granted that deference to a reliable source is the policy most likely to bring about right action, a refusal to defer carries with it a heightened risk of wronging and mistreating people. And that heightened risk of wrongdoing, I argue, cannot be justified by appeal to the intrinsic value of striving for a morally exemplary decision-making process.