Practicing theory building in a many modelers hackathon: A proof of concept

Dongen, Noah N'Djaye Nikolai van; Finnemann, Adam; Ron, Jill de; Tiokhin, Leonid; Wang, Shirley B.; Algermissen, Johannes; Altmann, Elena C.; Chuang, Li-Ching; Dumbravă, Andrei; Bahník, Štěpán; Fuenderich, Jens; Geiger, Sandra J.; Gerasimova, Daria; Golan, Aidai; Herbers, Judith; Lin, Yih-Shiuan; Moreau, David; Oberholzer, Yvonne; Peetz, Hannah Katharina; Rohrer, Julia M.; Rothers, Adrian; Schönbrodt, Felix D.; Seetahul, Yashvin; Szabelska, Anna; Tonge, Natasha April; Walasek, Nicole; Werner, Marlene; Borsboom, Denny

doi:10.31234/osf.io/r5yfz

Cited by 5 publications

(6 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Though the primary means by which theoreticians and empiricists interact is indirectly through citation of one another's publications, direct collaboration might be a more effective means for reducing barriers in understanding, though the relative scarcity of theoreticians in most biological fields can limit such direct interactions. Interestingly, a group of psychologists have successfully attempted an exercise called the 'Many Modelers Hackathon' where a few theoreticians worked with many empiricists to formalize verbal theories during a 3-hour workshop [69]. This Many Modelers event was successfully run as part of a 2021 conference for the Society for the Improvement of Psychological Science (SIPS) -a model for the Society for Open, Reliable, and…”

Section: Proposed Solutions: More Development Education and Collabora...mentioning

confidence: 99%

“…A second strategy is for societies to include hackathon-like activities in their conference schedules. More specifically, they could organize Many-Modeler-like hackathons where theoreticians and empiricists interface to formalize theories or create testable hypotheses, as discussed above [69]. It is important to note that people may believe that computer programming skills are a prerequisite for participation in a hackathon, but the term is now widely used for any event that is structured around team-based creative problem-solving.…”

Section: Idea: Inclusion Diversity Equity and Accessmentioning

confidence: 99%

See 1 more Smart Citation

Poor hypotheses and research waste in biology: learning from a theory crisis in psychology

Nakagawa,

Armitage,

Froese

et al. 2024

Preprint

View full text Add to dashboard Cite

While psychologists have extensively discussed a ‘theory crisis’, there has been no debate about such a crisis in biology. However, biologists, especially those working in the fields of ecology and evolution, have long discussed communication failures between theoreticians and empiricists. We argue such failure is one aspect of a theory crisis because misapplied and misunderstood theories lead to poor hypotheses and research waste. We review solutions for a theory crisis, comparing them with methodology-focused solutions proposed for a replication crisis. One neglected solution deserving further attention concerns the systematic mapping of theoretical models. We conclude by discussing how promoting inclusion, diversity, equity, and accessibility (IDEA) in theoretical biology could contribute to ameliorating breakdowns in the theory-empirical cycle in biology.

show abstract

Section: Proposed Solutions: More Development Education and Collabora...mentioning

confidence: 99%

Section: Idea: Inclusion Diversity Equity and Accessmentioning

confidence: 99%

Poor hypotheses and research waste in biology: learning from a theory crisis in psychology

Nakagawa,

Armitage,

Froese

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Journals can help to reduce the harms associated with ambiguities by encouraging the formalization of theory and transparent communication about known ambiguities (van Rooij, 2022;Jamieson & Pexman, 2020). They might consider publishing articles or special issues in which empirical researchers collaborate with modelers to formalize theories, or in which many modelers independently develop models of an influential verbal theory (van Dongen et al, 2022). In addition, journals can be explicit that they value transparency about ambiguity and encourage authors to be transparent about ambiguity and reviewers not to penalize this kind of transparency.…”

Section: Recommendationsmentioning

confidence: 99%

Strategic ambiguity in the social sciences

Frankenhuis,

Panchanathan,

Smaldino

2023

Soc. Psychol. Bull.

View full text Add to dashboard Cite

In the wake of the replication crisis, there have been calls to increase the clarity and precision of theory in the social sciences. Here, we argue that the effects of these calls may be limited due to incentives favoring ambiguous theory. Intentionally or not, scientists can exploit theoretical ambiguities to make support for a claim appear stronger than it is. Practices include theory stretching, interpreting an ambiguous claim more expansively to absorb data outside of the scope of the original claim, and post-hoc precision, interpreting an ambiguous claim more narrowly so it appears more precisely aligned with the data. These practices lead to the overestimation of evidence for the original claim and create the appearance of consistent support and progressive research programs, which may in turn be rewarded by journals, funding agencies, and hiring committees. Selection for ambiguous research can occur even when scientists act in good faith. Although ambiguity might be inevitable or even useful in the early stages of theory construction, scientists should aim for increased clarity as knowledge advances. Science benefits from transparently communicating about known ambiguities. To attain transparency about ambiguity, we provide a set of recommendations for authors, reviewers, and journals. We conclude with suggestions for research on how scientists use strategic ambiguity to advance their careers and the ways in which norms, incentives, and practices favor strategic ambiguity. Our paper ends with a simple mathematical model exploring the conditions in which high-ambiguity theories are favored over low-ambiguity theories, providing a basis for future analyses.

show abstract

“…Progress toward a more precise, mechanistic understanding of creative cognition cannot be made by modeling alone, but will require the cooperation of theorists, modelers, and experimenters (Hitchcock et al, 2022; van Dongen et al, 2022; Wiggins & Bhattacharya, 2014). How might greater integration between NCR and computational modeling look?…”

Section: Toward Greater Integration Between Ncr and Computational Mod...mentioning

confidence: 99%

Toward greater computational modeling in neurocognitive creativity research.

Lloyd-Cox,

Pickering,

Beaty

et al. 2023

Psychology of Aesthetics, Creativity, and the Arts

View full text Add to dashboard Cite

Creative cognition is the driving force behind all cultural and scientific progress. In recent years, the field of neurocognitive creativity research (NCR) has made considerable progress in revealing the neural and psychological correlates of creative cognition. However, a detailed understanding of how cognitive processes produce creative ideas, and how these processes interact differently across tasks and individuals, remains elusive. In this article, we argue that the increased adoption of computational modeling can help greatly in achieving this goal. While the verbal theories guiding NCR have evolved from broader accounts into more specific descriptions of neurocognitive processes, they remain more open to interpretation and harder to falsify than formal models. Translating theories into computational models can make them more concrete, accessible, and easier to compare, and helps researchers to develop causal hypotheses for how variation in cognitive factors leads to variation in creative outcomes. Currently, however, computational modeling of creativity is conducted almost entirely separately from NCR, and few attempts have been made to embody the cognitive theories of NCR in models that can simulate performance on common lab-based tasks. In this article, we discuss theories of creative cognition and how they might benefit from the wider adoption of formal modeling. We also examine recent computational models of creativity and how these might be improved and better integrated with NCR. Finally, we describe a pathway toward a mechanistic understanding of creative cognition through the integration of computational modeling, psychological theory, and empirical research, outlining an example model based on dual-process accounts. Public Significance StatementThis review argues that creativity research would benefit greatly from the wider adoption of computational modeling. We discuss how translating verbal theories of creative cognition into formal computational models can make them more rigorous, accessible, and communicable, and can highlight questions for future research. We examine previous models of creativity and explain how these can be improved to benefit our understanding of human creative cognition and the development of artificial creative systems.

show abstract

Practicing theory building in a many modelers hackathon: A proof of concept

Cited by 5 publications

References 9 publications

Poor hypotheses and research waste in biology: learning from a theory crisis in psychology

Poor hypotheses and research waste in biology: learning from a theory crisis in psychology

Strategic ambiguity in the social sciences

Toward greater computational modeling in neurocognitive creativity research.

Contact Info

Product

Resources

About