Leveraging Machine Learning to Automatically Derive Robust Decision Strategies from Imperfect Knowledge of the Real World

Abstract

Teaching people clever heuristics is a promising approach to improve decision-making under uncertainty. The theory of resource rationality makes it possible to leverage machine learning to discover optimal heuristics automatically. One bottleneck of this approach is that the resulting decision strategies are only as good as the model of the decision problem that the machine learning methods were applied to. This is problematic because even domain experts cannot give complete and fully accurate descriptions of the decisions they face. To address this problem, we develop strategy discovery methods that are robust to potential inaccuracies in the description of the scenarios in which people will use the discovered decision strategies. The basic idea is to derive the strategy that will perform best in expectation across all possible real-world problems that could have given rise to the likely erroneous description that a domain expert provided. To achieve this, our method uses a probabilistic model of how the description of a decision problem might be corrupted by biases in human judgment and memory. Our method uses this model to perform Bayesian inference on which real-world scenarios might have given rise to the provided descriptions. We applied our Bayesian approach to robust strategy discovery in two domains: planning and risky choice. In both applications, we find that our approach is more robust to errors in the description of the decision problem and that teaching the strategies it discovers significantly improves human decision-making in scenarios where approaches ignoring the risk that the description might be incorrect are ineffective or even harmful. The methods developed in this article are an important step towards leveraging machine learning to improve human decision-making in the real world because they tackle the problem that the real world is fundamentally uncertain.

Identifying resource-rational heuristics for risky choice.

“…This suggests that human decision making still has room for improvement, even when people’s cognitive constraints are taken into account. Our method could be used to provide feedback and teach people which heuristics to use and under what circumstances, in a manner that accounts for their cognitive limitations, providing a computationally informed path to improving human decision making (Becker et al, 2022; Callaway, Jain, et al, 2022; Consul et al, 2022; Mehta et al, 2022; Skirzyński et al, 2021).…”

Boosting Human Decision-making with AI-Generated Decision Aids

“…This suggests that our method is, in principle, applicable to a wide range of decision problems people face in the real world as long as the essential structure of those problems can be modeled within our general metalevel MDP framework (Griffiths et al, 2019;Callaway et al, 2022b;a). We have previously argued that this is the case for a wide range of common real-world decisions, such as purchasing decisions, hiring choices, investment decisions, deciding which charity to donate to, medical diagnosis, treatment planning, and credit approval decisions (Mehta et al, 2022;Skirzyński et al, 2021a;Callaway et al, 2022a).…”

“…However, our approach does not require that all of those assumptions are correct. To the contrary, the methods described here can be combined with recent advances that have made strategy discovery methods robust to errors in the model of the decision problems to be solved (Mehta et al, 2022). Moreover, our strategy discovery methods can also be extended to environments where the true state of affairs is unknown because it cannot be observed directly (Heindrich et al, 2022).…”

“…This procedure significantly improved people's planning skills when they interacted with a different, transfer environment. Subsequent work employed intelligent cognitive tutors that teach people by showing video demonstrations of the automatically discovered strategy Mehta et al, 2022). developed intelligent cognitive tutors that improved people's planning in very large versions of the environment illustrated in Fig.…”