Preventing undesirable behavior of intelligent machines

Thomas, Philip S.; Silva, Bruno Castro da; Barto, Andrew G.; Giguere, Stephen; Brun, Yuriy; Brunskill, Emma

doi:10.1126/science.aag3311

Cited by 118 publications

(82 citation statements)

References 145 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In a recent article published by Thomas PS et al, the application of a defined Seldon algorithm is described. The authors propose a framework for designing machine learning algorithms and show how it can be used to construct algorithms that provide their users with the ability to easily place limits on the probability that the algorithm will produce any specified undesirable behavior [23].…”

Section: Discussionmentioning

confidence: 99%

Artificial intelligence: Who is responsible for the diagnosis?

et al. 2020

View full text Add to dashboard Cite

The aim of the paper is to find an answer to the question "Who or what is responsible for the benefits and harms of using artificial intelligence in radiology?" When human beings make decisions, the action itself is normally connected with a direct responsibility by the agent who generated the action. You have an effect on others, and therefore, you are responsible for what you do and what you decide to do. But if you do not do this yourself, but an artificial intelligence system, it becomes difficult and important to be able to ascribe responsibility when something goes wrong. The manuscript addresses the following statements: (1) using AI, the radiologist is responsible for the diagnosis; (2) radiologists must be trained on the use of AI since they are responsible for the actions of machines; (3) radiologists involved in R&D have the responsibility to guide the respect of rules for a trustworthy AI; (4) radiologist responsibility is at risk of validating the unknown (black box); (5) radiologist decision may be biased by the AI automation; (6)risk of a paradox: increasing AI tools to compensate the lack of radiologists; (7) need of informed consent and quality measures. Future legislation must outline the contours of the professional's responsibility, with respect to the provision of the service performed autonomously by AI, balancing the professional's ability to influence and therefore correct the machine, limiting the sphere of autonomy that instead technological evolution would like to recognize to robots.

show abstract

Section: Discussionmentioning

confidence: 99%

Artificial intelligence: Who is responsible for the diagnosis?

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Techniques have been developed to reduce social bias in some neural networks designs such as adjusting the high dimensional vectors representing individual words to remove differences in the distance from the concepts of male and female in word2vec [11] and the Seldonian approach of describing and regulating undesirable behavior [12].…”

Section: Related Workmentioning

confidence: 99%

The Real-World-Weight Cross-Entropy Loss Function: Modeling the Costs of Mislabeling

Ho¹,

Wookey²

2020

IEEE Access

557

209

View full text Add to dashboard Cite

In this paper, we propose a new metric to measure goodness-of-fit for classifiers: the Real World Cost function. This metric factors in information about a real world problem, such as financial impact, that other measures like accuracy or F1 do not. This metric is also more directly interpretable for users. To optimize for this metric, we introduce the Real-World-Weight Cross-Entropy loss function, in both binary classification and single-label multiclass classification variants. Both variants allow direct input of real world costs as weights. For single-label, multiclass classification, our loss function also allows direct penalization of probabilistic false positives, weighted by label, during the training of a machine learning model. We compare the design of our loss function to the binary cross-entropy and categorical cross-entropy functions, as well as their weighted variants, to discuss the potential for improvement in handling a variety of known shortcomings of machine learning, ranging from imbalanced classes to medical diagnostic error to reinforcement of social bias. We create scenarios that emulate those issues using the MNIST data set and demonstrate empirical results of our new loss function. Finally, we discuss our intuition about why this approach works and sketch a proof based on Maximum Likelihood Estimation.

show abstract

“…This group recommended that the development, deployment, and use of AI systems should adhere to the ethical principles of respect for human autonomy, prevention of harm, fairness/equity and explicability [27]. It is clear that an intrinsically ethical AI could be a solution to avoid some drawbacks, i.e., the case of the Seldon algorithm which places limits to the probability that an algorithm will produce any specified undesirable behaviour [28].…”

Section: Solutions: the Algor-ethicsmentioning

confidence: 99%

Artificial Intelligence in Radiology—Ethical Considerations

Brady

Neri

2020

Diagnostics

View full text Add to dashboard Cite

Artificial intelligence (AI) is poised to change much about the way we practice radiology in the near future. The power of AI tools has the potential to offer substantial benefit to patients. Conversely, there are dangers inherent in the deployment of AI in radiology, if this is done without regard to possible ethical risks. Some ethical issues are obvious; others are less easily discerned, and less easily avoided. This paper explains some of the ethical difficulties of which we are presently aware, and some of the measures we may take to protect against misuse of AI.

show abstract

Preventing undesirable behavior of intelligent machines

Cited by 118 publications

References 145 publications

Artificial intelligence: Who is responsible for the diagnosis?

Artificial intelligence: Who is responsible for the diagnosis?

The Real-World-Weight Cross-Entropy Loss Function: Modeling the Costs of Mislabeling

Artificial Intelligence in Radiology—Ethical Considerations

Contact Info

Product

Resources

About