Overcoming Learning Aversion in Evaluating and Managing Uncertain Risks

Abstract

Decision biases can distort cost-benefit evaluations of uncertain risks, leading to risk management policy decisions with predictably high retrospective regret. We argue that well-documented decision biases encourage learning aversion, or predictably suboptimal learning and premature decision making in the face of high uncertainty about the costs, risks, and benefits of proposed changes. Biases such as narrow framing, overconfidence, confirmation bias, optimism bias, ambiguity aversion, and hyperbolic discounting of the immediate costs and delayed benefits of learning, contribute to deficient individual and group learning, avoidance of information seeking, underestimation of the value of further information, and hence needlessly inaccurate risk-cost-benefit estimates and suboptimal risk management decisions. In practice, such biases can create predictable regret in selection of potential risk-reducing regulations. Low-regret learning strategies based on computational reinforcement learning models can potentially overcome some of these suboptimal decision processes by replacing aversion to uncertain probabilities with actions calculated to balance exploration (deliberate experimentation and uncertainty reduction) and exploitation (taking actions to maximize the sum of expected immediate reward, expected discounted future reward, and value of information). We discuss the proposed framework for understanding and overcoming learning aversion and for implementing low-regret learning strategies using regulation of air pollutants with uncertain health effects as an example.

Muddling-Through and Deep Learning for Managing Large-Scale Uncertain Risks

“…For example, finding that investments in a costly course of action have yielded lower-than-expected returns may provoke those who originally chose it to escalate their commitment to it (Molden & Hui, 2011; Schultze et al, 2012). Possible psychological and political explanations for escalating commitment range from loss aversion to seeking to manage the impressions of others, but clearly such resistance to modifying or abandoning previous choices in light of experience inhibits effective learning (Cox, 2015; Tetlock & Gardner, 2015). In business as well as government, data needed to evaluate and compare actual to predicted performance of a policy are often not even collected, or are ignored or misinterpreted if they are collected (Russo & Schoemaker, 1989).…”

“…Reasons abound in individual and group psychology for keeping those who make decisions about policy adjustments (analogous to “actors” in actor-critic RL algorithms) separate from those who evaluate the performance of the policies and provide feedback and suggestions for improving them (the “critics”). Among these reasons are confirmation bias, motivated reasoning, groupthink, and other heuristics and biases (Cox, 2015). RL suggests an additional reason, rooted in statistics: in deep learning RL algorithms, training one network to decide what to do next and a separate one to evaluate how well it is working has been found to prevent overly optimistic assessments of policy performance due to overfitting, i.e., using the same data to both select estimated value-maximizing actions and estimate the values from taking those actions (van Hesselt et al, 2015).…”

“…States and counties implementing pollution-reducing regulations might learn from each other's experiences about which combinations of interventions and conditions (possibly involving copollutants, weather variables, and sociodemographic characteristics of the exposed population) generate the greatest public health benefits from pollution reduction measures. As usual, effective learning in human organizations must overcome challenges from various types of learning aversion that have no clear counterparts in machine learning (Cox, 2015). For example, human bureaucracies may reorganize to visibly mimic organizational structures in more successful organizations whose reputations they covet, but without corresponding learning of the more effective policies that drive improved performance (Monios, 2016).…”

Learning and Dynamic Choices Under Uncertainty: From Weighted Regret and Rejoice to Expected Utility

“…Needless to say, the findings presented in this paper require further investigation, both within the same contexts (e.g., by allowing the degree of risk aversion to be updated according to experience, or the degree of regret to differ from the degree of rejoice) and in alternative contexts. For example, research should consider unknown probability distributions (Palley and Kremer, 2014), unobserved outcomes of foregone alternatives, the additional possibility of choosing actions for exploration (Gonzalez, 2013;Cox, 2015), and learning from other people's experience.…”

Learning and dynamic choices under uncertainty: From weighted regret and rejoice to expected utility

“…Needless to say, the findings presented in this paper require further investigation in alternative contexts. For example, research should consider scenarios with unknown probability distributions (Palley & Kremer, 2014), unobserved outcomes of alternatives that were not chosen (Mengel & Rivas, 2012), the additional possibility of choosing actions for exploration (Cox, 2015;Gonzalez, 2013), and learning from other people's experience (Meub & Proeger, 2017).…”