Promises and perils of experimentation: The mutual internal validity problem

Abstract

Experimentation is the cornerstone of the scientific method. Decades of experimental work in psychology, however, has yet to produce reliable and cumulative knowledge. We argue psychology as a discipline not only overuses experimentation, but too often uses it as a theatrical storytelling device. Rather than use experiments to facilitate valid scientific inference, psychologists cleverly design experiments to elicit the very effects they wish to observe, ignoring how these effects may or may not cohere with bodies of knowledge within psychology and across disciplines. This approach leads to ungeneralizable results and an unwillingness to accept and learn from failed, but potentially informative experiments. To make real progress toward understanding human behavior and the mind, psychology needs to embrace big-interdisciplinary (big-I) triangulation, which requires integrating theories, methods, and approaches from diverse disciplines and sources.

Self-control: An integrative framework

“…This fragmentation is understandable -when different subfields publish their findings in different journals and use different terms to refer to the same construct (i.e., jangle fallacy; Kelley, 1927), it can be challenging to find the points of commonality. But we propose that addressing these barriers and integrating across these fields is crucial for building a more comprehensive and cumulative science (e.g., Lin et al, 2020). While part of our aim in presenting the current framework was to integrate across different theories and perspectives, by no means is this a complete endeavour and so here we highlight a few notable points to consider as we move toward building a more cumulative approach to studying selfcontrol in everyday life and across different domains.…”

A decade of theory as reflected in Psychological Science (2009-2019)

“…One possibility is that the research is truly a-theoretical or descriptive. There is clear value in descriptive research-value that can ultimately amount to theoretical advancement (17,18,20) and it would be misguided to avoid interesting questions because they did not originate from theory.…”

“…One factor contributing to this dilemma is that theories are often vague and poorly specified (2,15) , so a given theory is unable to adequately explain a range of phenomena without relying on rhetoric. Thus, psychology uses experimentation to tell a narrative rather than to test theoretical predictions (16,17). From this perspective, psychology needs more exploratory and descriptive research before moving on to theory building and testing (18)(19)(20).…”

Do early birds share their worms? How empathy and prosociality vary across the course of the day

“…Furthermore, we were not able to distinguish between different types of prosocial activities; it is possible that some types of prosocial behaviours are affected by circadian rhythms or synchrony effects more substantially than others. Future research should continue to seek a balance between the ecological validity and generalizability afforded by experience-sampling studies and the controlled internal validity afforded by lab-based experiments (Lin et al, 2020).…”