Why Providing Humans with Interpretable Algorithms May, Counterintuitively, Lead to Lower Decision-making Performance

DeStefano, Timothy; Kellogg, Katherine C.; Menietti, Michael; Vendraminelli, Luca

doi:10.2139/ssrn.4246077

Cited by 5 publications

(5 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, such computational overconfidence might lead to insufficient utilization of better-performing decision-support models. Similar resistance to an interpretable algorithm in the context of repetitive decision-making under uncertainty has been recently reported by DeStefano et al (2022). Their novel field experiment was conducted at a retail company while it was switching its stock reordering decision support system from an interpretable (weighted moving average) model to an uninterpretable (recurrent neural network) model.…”

Section: Discussionsupporting

confidence: 55%

The Effect of Interpretable Artificial Intelligence on Repeated Managerial Decision-Making Under Uncertainty

Altintas

2023

SSRN Journal

View full text Add to dashboard Cite

Section: Discussionsupporting

confidence: 55%

The Effect of Interpretable Artificial Intelligence on Repeated Managerial Decision-Making Under Uncertainty

Altintas

2023

SSRN Journal

View full text Add to dashboard Cite

“…In fact, empirical studies have found XAI detrimental in uncertain environments, as humans are more likely to reject helpful recommendations because of overconfidence in their troubleshooting abilities. 202 In many cases, automatic identification of anomalies (vide supra) for review by a human operator suffices as long as the anomalies are rare. The human scientist can then invoke their own reasoning, statistical evidence, or other forms of investigation to study the problem.…”

Section: Vc Sample What Can Be Made and How To Make It � Defer Optimi...mentioning

confidence: 99%

“…Moreover, the most appropriate models for initial discoveryfor both interpretability and extrapolationmay be the types of feature-selected linear models discussed above, obviating the need for more sophisticated black-box model interpretability methods. In fact, empirical studies have found XAI detrimental in uncertain environments, as humans are more likely to reject helpful recommendations because of overconfidence in their troubleshooting abilities . In many cases, automatic identification of anomalies ( vide supra ) for review by a human operator suffices as long as the anomalies are rare.…”

Section: Recommendations Toward ML For Exceptional Materialsmentioning

confidence: 99%

In Pursuit of the Exceptional: Research Directions for Machine Learning in Chemical and Materials Science

Schrier,

Norquist,

Buonassisi

et al. 2023

J. Am. Chem. Soc.

View full text Add to dashboard Cite

Exceptional molecules and materials with one or more extraordinary properties are both technologically valuable and fundamentally interesting, because they often involve new physical phenomena or new compositions that defy expectations. Historically, exceptionality has been achieved through serendipity, but recently, machine learning (ML) and automated experimentation have been widely proposed to accelerate target identification and synthesis planning. In this Perspective, we argue that the data-driven methods commonly used today are well-suited for optimization but not for the realization of new exceptional materials or molecules. Finding such outliers should be possible using ML, but only by shifting away from using traditional ML approaches that tweak the composition, crystal structure, or reaction pathway. We highlight case studies of high-T c oxide superconductors and superhard materials to demonstrate the challenges of ML-guided discovery and discuss the limitations of automation for this task. We then provide six recommendations for the development of ML methods capable of exceptional materials discovery: (i) Avoid the tyranny of the middle and focus on extrema; (ii) When data are limited, qualitative predictions that provide direction are more valuable than interpolative accuracy; (iii) Sample what can be made and how to make it and defer optimization; (iv) Create room (and look) for the unexpected while pursuing your goal; (v) Try to fill-in-the-blanks of input and output space; (vi) Do not confuse human understanding with model interpretability. We conclude with a description of how these recommendations can be integrated into automated discovery workflows, which should enable the discovery of exceptional molecules and materials.

show abstract

“…In fact, empirical studies have found XAI detrimental in uncertain environments, as humans are more likely to reject helpful recommendations because of overconfidence in their troubleshooting abilities. 202 In many cases, automatic identification of anomalies (vide supra) for review by a human operator suffices, so long as the anomalies are rare. The human scientist can then invoke their own reasoning, statistical evidence, or other forms of investigation to study the problem.…”

Section: Try To Fill-in-the-blanks Of Input and Output Spacementioning

confidence: 99%

In Pursuit of the Exceptional: Research Directions for Machine Learning in Chemical and Materials Science

Schrier,

Norquist,

Buonassisi

et al. 2023

Preprint

View full text Add to dashboard Cite

Exceptional molecules and materials with one or more extraordinary properties are both technologically valuable and fundamentally interesting because they often involve new physical phenomena or new compositions that defy expectations. Historically, exceptionality has been achieved through serendipity, but recently, machine learning (ML) and automated experimentation have been widely proposed to accelerate target identification and synthesis planning. In this Perspective, we argue that the data-driven methods commonly used today are well-suited for optimization but not for realizing new exceptional materials or molecules. Finding such outliers should be possible using ML, but only by shifting away from using traditional ML approaches that tweak the composition, crystal structure, or reaction pathway. We highlight case studies of high-Tc oxide superconductors and superhard materials to demonstrate the challenges of ML-guided discovery and discuss the limitations of automation for this task. We then provide six recommendations for the development of ML methods capable of exceptional materials discovery: (i) Avoid the tyranny of the middle and focus on extrema; (ii) When data is limited, qualitative predictions that provide direction are more valuable than interpolative accuracy; (iii) Sample what can be made and how to make it, and defer optimization; (iv) Create room (and look) for the unexpected while pursuing your goal; (v) Try to fill-in-the-blanks of input and output space; (vi) Do not confuse human understanding with model interpretability. We conclude with a description of how these recommendations can be integrated into automated discovery workflows that should enable the discovery of exceptional molecules and materials.

show abstract

Why Providing Humans with Interpretable Algorithms May, Counterintuitively, Lead to Lower Decision-making Performance

Cited by 5 publications

References 24 publications

The Effect of Interpretable Artificial Intelligence on Repeated Managerial Decision-Making Under Uncertainty

The Effect of Interpretable Artificial Intelligence on Repeated Managerial Decision-Making Under Uncertainty

In Pursuit of the Exceptional: Research Directions for Machine Learning in Chemical and Materials Science

In Pursuit of the Exceptional: Research Directions for Machine Learning in Chemical and Materials Science

Contact Info

Product

Resources

About