Generalizability is not optional: insights from a cross-cultural study of social discounting

Abstract

Current scientific reforms focus more on solutions to the problem of reliability (e.g. direct replications) than generalizability. Here, we use a cross-cultural study of social discounting to illustrate the utility of a complementary focus on generalizability across diverse human populations. Social discounting is the tendency to sacrifice more for socially close individuals—a phenomenon replicated across countries and laboratories. Yet, when adapting a typical protocol to low-literacy, resource-scarce settings in Bangladesh and Indonesia, we find no independent effect of social distance on generosity, despite still documenting this effect among US participants. Several reliability and validity checks suggest that methodological issues alone cannot explain this finding. These results illustrate why we must complement replication efforts with investment in strong checks on generalizability. By failing to do so, we risk developing theories of human nature that reliably explain behaviour among only a thin slice of humanity.

Learning from failures of protocol in cross-cultural research

“…The apparent regularity and pervasiveness of social discounting have even led one set of researchers to call it the "inverse distance law of giving" (49) and others to identify its underlying neural basis (50). A growing database of more than 50 studies, including a preregistered direct replication, has reliably documented social discounting (51). However, current studies rely almost exclusively on highly educated populations in the United States and Europe, with the addition of some highly educated populations in China (52), India (53), and Singapore (54).…”

“…The "amount foregone to give someone $75" is then calculated as the average amount foregone between the responses where the respondent crossed from A to B. Those rare cases (usually not more than 10% of participants) that have more than one crossover point are conventionally excluded from analyses (46,51,57). With an estimate of the amount foregone to give someone $75, a researcher can then examine how this varies across partners at varying social distances.…”

“…In the social discounting literature, researchers frequently discard these participants or choice sets (51). This decision relies on a model of responding whereby respondents treat each decision as independent of the others and where they apply the same utility function to each independent decision.…”

“…These assumptions are not particularly problematic in the vast majority of discounting studies among college students. In most cases, fewer than 10% of participants are discarded, with the underlying assumption that those participants did not understand the task or were not paying attention (51).…”

Cultural Evolution of Genetic Heritability

“…Each individual experiences just a snapshot, making the global mechanics opaque. Instead, each of us is left with an intuition that our world is largely the world, which perhaps explains why the WEIRD people problem is underappreciated even a decade after publication of Henrich et al (2010) (Nielsen et al 2017;Pollet and Saxton 2019;Tiokhin et al 2019;Barrett 2020). From this limited vantage point, we evaluate questions such as the relative contributions of nature versus nurture.…”