A Call for Greater Modesty in Psychology and Cognitive Neuroscience

Abstract

Exaggerated claims and low levels of reproducibility are commonplace in psychology and cognitive neuroscience, due to an incentive structure that demands “newsworthy” results. My overall argument here is that in addition to methodological reform, greater modesty is required across all levels - from individual researchers to the systems that govern science (e.g., editors, reviewers, grant panels, hiring committees) - to redirect expectations regarding what psychological and brain science can effectively deliver. Empirical work and the reform agenda should pivot away from making big claims on narrow evidence bases or single tools and focus on the limitations of our individual efforts, as well as how we can work together to build ways of thinking that enable integration and synthesis across multiple modalities and levels of description. I outline why modesty matters for science including the reform agenda, provide some practical steps that we can take to embrace modesty, rebut common misconceptions of what modesty means for science, and present some limitations of the approach. Ultimately, by presenting a more sober view of our capacities and achievements, whilst placing work within a wider context that respects the complexity of the human brain, we will bolster the fidelity of scientific inference and thus help in a small way to generate a firmer footing upon which to build a cumulative science.

Integrating Social Cognition Into Domain‐General Control: Interactive Activation and Competition for the Control of Action (ICON)

“…As argued by Simons, Shoda, and Lindsay (2017), we feel it is valuable to explicitly state the limitations of our approach, and likely constraints on the generality of our findings, to provide a more stable and transparent platform for future research (Ramsey, 2021).…”

The promise and pitfalls of studying the neurophysiological correlates of automatic imitation

“…However, until preregistered replication studies have been performed that use well-justified sample sizes and statistical approaches, we should remain very cautious about these results, given the wider landscape of irreproducible research and questionable research practices that plagues psychology and human neuroscience research in general (Munafò et al, 2017;Nosek et al, 2022;Simmons et al, 2018). As such, just like any other aspect of science, it is essential that modest claims are drawn before the reliability, validity, and generalisability of measures and findings can be evidenced (Ramsey, 2021).…”

“…It would have been extremely valuable to show in comprehensive follow-up research that these initial findings were robust and that the measurement tools were reliable and licensed valid inferences. As such, we think that researchers should adopt a more cautious attitude to science, especially when evaluating novel findings (Ramsey, 2021).…”

Integrating Social Cognition into Domain-General Control: Interactive activation and competition for the CONtrol of action (ICON)

“…Limitations. As argued by Simons et al (2017), we feel it is valuable to explicitly state limitations of our approach, and likely constraints on the generality of our findings, to provide a more stable and transparent platform for future research (Ramsey, 2021).…”