Mixed‐effects regression weights for advice taking and related phenomena of information sampling and utilization

Abstract

Advice taking and related research is dominated by deterministic weighting indices, specifically ratio‐of‐differences‐based formulas for investigating informational influence. Their arithmetic is intuitively simple, but they pose several measurement problems and restrict research to a particular paradigmatic approach. As a solution, we propose to specify how strongly peoples' judgments are influenced by externally provided evidence by fitting corresponding mixed‐effects regression models. Our approach explicitly distinguishes between endogenous components, such as updated beliefs, and exogenous components, such as independent initial judgments and advice. Crucially, mixed‐effects regression coefficients of various exogenous sources of information also reflect individual weighting but are based on a conceptually consistent representation of the endogenous judgment process. The formal derivation of the proposed weighting measures is accompanied by a detailed elaboration on their most important technical and statistical subtleties. We use this modeling approach to revisit empirical findings from several paradigms investigating algorithm aversion, sequential collaboration, and advice taking. In summary, we replicate and extend the original finding of algorithm appreciation and initially demonstrate a lack of evidence for both systematic order effects in sequential collaboration as well as differential weighting of multiple pieces of advice. In addition to opening new avenues for innovative research, appropriate modeling of information sampling and utilization has the potential to increase the reproducibility and replicability of behavioral science. Furthermore, the proposed method is relevant beyond advice taking, as mixed‐effects regression weights can also inform research on related cognitive phenomena such as multidimensional belief updating, anchoring effects, hindsight bias, or attitude change.

Modeling Dependent Group Judgments: A Computational Model of Sequential Collaboration

“…The result that individuals do not ignore presented judgments stands in contrast to findings of advice taking literature which typically showed egocentric discounting of advice (Bonaccio & Dalal, 2006). Moreover, Rebholz et al (2024) did not find any differences in the extent of advice taking over the course of a sequential chain even though the variability in the extent of integration of the presented judgment decreases over the course of a sequential chain.…”

“…The idea of a general response bias is a common assumption in many modeling approaches such as signal detection theory which aims at disentangling an individual's detection ability from their response criterion (Lynn & Barrett, 2014;Pastore & Scheirer, 1974). Similarly, in judgment and decision making, individuals differ in the extent of accepting advice (Bonaccio & Dalal, 2006;Rebholz et al, 2024) and the extent of showing an anchoring bias (Cheek & Norem, 2022;Tversky & Kahneman, 1974).…”

Investigating Factors Influencing Audiences’ Integration of Scientific Evidence

“…Belief updating was assessed using the mixed-effects regression approach by Rebholz et al (2024). The posterior beliefs P o ij of participants i = 1, .…”

“…where the two equations specify behavior on the judgment and weighting level, respectively (Rebholz et al, 2024). The implemented sum-to-one constraining implied complementary weighting of participants' prior beliefs denoted as P r ij and the presented evidence ratio denoted as E ij .…”