A deep-learning model of prescient ideas demonstrates that they emerge from the periphery

Abstract

Where do prescient ideas—those that initially challenge conventional assumptions but later achieve widespread acceptance—come from? Although their outcomes in the form of technical innovation are readily observed, the underlying ideas that eventually change the world are often obscured. Here we develop a novel method that uses deep learning to unearth the markers of prescient ideas from the language used by individuals and groups. Our language-based measure identifies prescient actors and documents that prevailing methods would fail to detect. Applying our model to corpora spanning the disparate worlds of politics, law, and business, we demonstrate that it reliably detects prescient ideas in each domain. Moreover, counter to many prevailing intuitions, prescient ideas emanate from each domain's periphery rather than its core. These findings suggest that the propensity to generate far-sighted ideas may be as much a property of contexts as of individuals.

Judicial hierarchy and discursive influence

“…We have found that the US Supreme Court influences legal discourse—a meso-level phenomenon between the micro-level of legal rules and the macro-level of legal subject areas—both through its ability to select cases through review as well as (to a more limited extent) its own innovations. There is a robust literature that examines the relationship between the ‘core’ and ‘periphery’ in the cultural context [ 24 ], and whether innovation tends to occur within the well-connected, and often powerful centre, or at the edges of a social network, which may be less constrained and have more space for experimentation [ 19 ]. Our findings indicate that the Supreme Court—as the most core institution within the U.S. judiciary—plays an important role in influencing judicial discourse, but it does so not through its own innovation, but through its identification of innovation closer to the periphery.…”

“…Some approaches seek to identify innovation at the level of concepts; for this task, identifying new terms or usages is especially important [ 18 ]. Alternatively, innovation can be understood as surprising combinations of words, where surprise could be estimated based on a predictive model [ 19 ]. For our investigation we apply a Document Influence Model (DIM) [ 5 , 6 ].…”

Mining for Meaning : using computational text analysis for social inquiry

“…Recently, a class of complex nonlinear models-sometimes referred to as deep learning models, or "black-box" modelshas been gaining momentum in social science. These involve an increased use of pre-trained transformer models Vicinanza et al 2023;Card et al 2022;Le Mens et al 2023;Kovács et al 2023), which are then typically fine-tuned to fit a specific research question. For example, used theory to label which texts were about "nationalism" and "populism," which they then used to fine-tune a model to automatically identify these concepts in other texts.…”

Minority-group incubators and majority-group reservoirs support the diffusion of climate change adaptations

“…In the social-movements literature, there is clear evidence that heterogeneity facilitates diffusion, though there is some conflicting evidence whether innovations arising in the core [18] or the periphery [19] are more likely to lead to a cascade of adoption. Transient diversity, suggesting heterogeneity in knowledge networks, can support problem solving [20,21] and prescient ideas often emerge from the peripheries of metapopulations [22]. Intriguingly, studies of the evolution of anti-microbial resistance (an acultural system) suggest that it often occurs in peripheral groups in metapopulations [23,24].…”