Representative design in psychological assessment: A case study using the Balloon Analogue Risk Task (BART).

Abstract

Representative design refers to the idea that experimental stimuli should be sampled or designed such that they represent the environments to which measured constructs are supposed to generalize. In this article we investigate the role of representative design in achieving valid and reliable psychological assessments, by focusing on a widely used behavioral measure of risk taking-the Balloon Analogue Risk Task (BART). Specifically, we demonstrate that the typical implementation of this task violates the principle of representative design, thus conflicting with the expectations people likely form from real balloons. This observation may provide an explanation for the previously observed limitations in some of the BART's psychometric properties (e.g., convergent validity with other measures of risk taking). To experimentally test the effects of improved representative designs, we conducted two extensive empirical studies (N = 772 and N = 632), finding that participants acquired more accurate beliefs about the optimal behavior in the BART due to these task adaptions. Yet, improving the task's representativeness proved to be insufficient to enhance the BART's psychometric properties. It follows that for the development of valid behavioral measurement instruments-as are needed, for instance, in functional neuroimaging studies-our field has to overcome the philosophy of the "repair program" (i.e., fixing existing tasks). Instead, we suggest that the development of valid task designs requires novel ecological assessments, aimed at identifying those real-life behaviors and associated psychological processes that lab tasks are supposed to capture and generalize to.

Bursting balloons - comparison of risk taking between extreme sports, esports, and the general public

“…Further, when asked to gauge their general willingness to take risks, it heavily depends on which domain extreme sports athletes have on their minds (Hanoch et al, 2006). In contrast, behavioral risk-taking measures benefit from eliciting risk taking in a more or less natural environment (Steiner & Frey, 2021) independent of beliefs about domain-specific risk taking. One of the most prominent behavioral measures of risk taking, the Balloon Analogue Risk Task (BART), has been used across various experimental settings (e.g., brain research, Sehrig et al, 2021;lab studies, Lauriola et al, 2014), with different groups of participants (e.g., adolescents, Aklin et al, 2005;older adults, Rolison et al, 2012), and to investigate diverse research questions.…”

Bursting Balloons - Comparison of Risk Propensity Between Extreme Sports Esports and the General Public

“…Further, when asked to gauge their general willingness to take risks, it heavily depends on which domain extreme sports EXTREME SPORTS & RISK TAKING IN THE BART 6/30 athletes have on their minds (Hanoch et al, 2006). In contrast, behavioral risk-taking measures benefit from eliciting risk taking in a more or less natural environment (Steiner & Frey, 2021) independent of beliefs about domain-specific risk taking. One of the most prominent behavioral measures of risk taking, the Balloon Analogue Risk Task (BART), has been used across various experimental settings (e.g., brain research, Sehrig et al, 2020;lab studies, Lauriola et al, 2014), with different groups of participants (e.g., adolescents, Aklin et al, 2005; older adults, Rolison et al, 2012), and to investigate diverse research questions.…”

The assessment of physical risk taking: Preliminary construct validation of a new behavioral measure

“…Unfortunately, manipulating factors beyond loss probability and reward amount in the BART itself is challenging (e.g., loss magnitude cannot be altered much as the balloon either explodes, which results in loss of money, or it does not and more money is earned). Moreover, recent research has also criticized the representative design of the BART, suggesting that “fixing” or altering the task in some way to improve it may not be the best path forward to study risky decision making and the factors influencing it [ 18 ]. Instead, new tasks that inherently possess high representative design should be created and implemented.…”