Negative Affect Induces Rapid Learning of Counterfactual Representations: A Model-based Facial Expression Analysis Approach

Abstract

Regret-an emotion comprised of both a counterfactual, cognitive component and a negative, affective component-is one of the most commonly experienced emotions involved in decision making. For example, people often behave such that their decisions minimize potential regret and therefore maximize subjective pleasure. Importantly, functional accounts of emotion suggest that the experience and future expectation of regret should promote goal-oriented behavioral change. While many studies have confirmed the functional role of regret, the cognitiveemotional mechanisms through which regret facilitates behavioral change remain unclear. We hypothesize that a greater negative affective state accompanying the counterfactual component of regret will potentiate learning, value, and/or exploration of potential options-all of which could lead to behavioral change. Because prior studies on regret-driven decision making have focused almost exclusively on description-based paradigms, the potential role of learning from regret through experience is underexplored. Here, we leverage computational, emotion-driven models of decision making to determine the role of regret in risky decision making. Further, we use computer-vision to detect positive and negative affect (valence) intensity from subjects' faces in response to feedback, which was entered as direct input into each model to determine whether valence affects learning, valuation, or exploration/exploitation. Using multiple model comparison methods, we found that: (1) people weight regret by its expectedness when making experience-based decisions, and (2) people learn more rapidly as they experience increasingly intense negative affect. Our findings support functional accounts of regret and demonstrate the potential for model-based facial expression analysis to enhance our understanding of cognitionemotion interactions.

Regret in experience-based decisions: The effects of expected value differences and mixed gains and losses.

“…In the present study, we found that when making decisions from experience, individuals are indeed risk seeking overall if the riskier and safer options have equal EVs and all outcomes are gains, conceptually replicating prior work (Ahn et al, 2012; Haines et al, 2021). Because the riskier option occasionally produced low outcomes that should have elicited the strongest feelings of regret in our task, this finding conflicts with the assumption of a convex Q (.)…”

“…In summary, although there is a general tendency to choose options that minimize the probability of regret in experience-based decisions, structural features of the choice scenario, such as EV differences and the sign of outcomes, can diminish this tendency. More generally, our results add to a growing body of research suggesting that relative comparisons between obtained and forgone outcomes play a key role in guiding decision-making (Ahn et al, 2012; Coricelli et al, 2007; Haines et al, 2021;, 1999). However, our findings diverge from standard regret theory applied to decisions from the description in which Q (.)…”

“…Importantly, the two options always had equal EVs but the probability that the risky option yielded a higher reward increased from .20 to .40 to .60 to .80 across four separate games. Selections of the risky option increased as its probability of yielding a higher reward increased, and computational modeling suggested that participants sought to avoid making choices that had a high expected probability of resulting in regret (Ahn et al, 2012; see also Haines et al, 2021). In the choice between the hypothetical gambles R and S discussed above, minimizing the probability of regret would lead to a preference for R since it produces a better outcome than S on 80% of draws.…”

Using computer-vision and machine learning to automate facial coding of positive and negative affect intensity

“…Decades of research show clear links between facial expressions of emotion and cognitive processes in aggregate (see [56,57]), yet the dynamics between cognitive mechanisms and facial expressions are poorly understood in part due to difficulties accompanying manual coding. In fact, we are currently using computational modeling to explore cognition-expression relationships with the aid of CVML [58], which would be infeasible with manual coding of facial expressions. For example, in the current study it took less than three days to automatically extract AUs from 4,648 video recordings and train ML models to generate valence intensity ratings (using a standard desktop computer).…”

Using Computer-vision and Machine Learning to Automate Facial Coding of Positive and Negative Affect Intensity

“…Decades of research show clear links between facial expressions of emotion and cognitive processes in aggregate (see 51,52), yet the dynamics between cognitive mechanisms and facial expressions are poorly understood in part due to difficulties accompanying manual coding. In fact, we are currently using computational modeling to explore cognition-expression relationships with the aid of CVML (53), which would be infeasible with manual coding of facial expressions. For example, in the current study it took less than three days to automatically extract AUs from 4,648 video recordings and train ML models to generate valence intensity ratings (using a standard desktop computer).…”