Generalizations: The grail and the gremlins.

Abstract

The current state of psychological science is such that we frequently fail to achieve one of the major goals of research: to generalize findings beyond the samples, stimuli, and paradigms on which they were observed. Generalizations are constrained by restricted and nonrepresentative samples, biased operationalizations and overbroad characterizations, stimulus and paradigmatic threats to construct and ecological validity, and overreliance on mean differences at the expense of attention to individual variability. I illustrate these threats to generalization using examples from cognitive research. I contend that these practices lessen the authenticity of our research and if left unchecked, will render our discipline irrelevant and obsolete, a threat as existential as that posed by practices that limit rigor and replicability. I make suggestions for ways to improve current research practices that will help to build a more valid, generalizable, and applicable science, thus advancing both basic and applied research in memory and cognition.

If generalization is the grail, practical relevance is the nirvana: Considerations from the contribution of psychological science of memory to law.

“…When applying memory research to reallife examples, a pivotal issue is which research can be used for this application. Bauer (2023) argued that psychological science is failing at one of its cornerstones: delivering generalizable knowledge. Specifically, a large portion of psychological literature is constrained in making generalizable claims because it involves nonrepresentative samples, overreliance on mean differences, threats to measurement and ecological validity, and stimulus paucity, among other issues.…”

Gremlins in childhood amnesia research.

“… Bauer (2023) has provided a timely reminder that psychological research findings must be able to be generalized beyond the samples, stimuli, and paradigms used. In this commentary, I illustrate how research in childhood amnesia falls short.…”

“…Bauer (2023) has reminded us that although generalizability of research results is a fundamental goal of our field, we are falling short in many ways. Her article points out a number of issues with how we often currently do research that limits our ability to generalize our findings—issues that she calls gremlins.…”

“…Overall, she has presented an excellent and timely "call to action" which she hopes will serve as a catalyst for changes in both what we do and how we report our findings. Bauer (2023) provides a description of a research program conducted in her lab on how people build semantic knowledge; she then uses that research as an example of problems with the generalizability of findings. Among the problems she points out, the research had been based on specific samples of participants who are not representative of many other groups of people, not to mention the huge individual variability between participants.…”

“…A host of research has identified the average age of earliest memory in adults as between 3 and 4 years of age (often stated as averaging age 3½ years), and this is almost universally reported in textbooks when they talk about this phenomenon. It has played a role in theory development (Bauer, 2007;Nelson & Fivush, 2004;Perner & Ruffman, 1995;Pillemer & White, 1989;Reese, 2009;Wang, 2003) as well as has been generalized to forensic situations (Bauer, 2023). As Bauer notes, the lack of memories from early life in adults was assumed to mean that children had such rudimentary memory and linguistic systems at that age that they could not form memories.…”