Expert opinions on the smallest effect size of interest in false memory research

Abstract

In the present study, we used a new approach to establish the smallest effect size of interest (SESOI) for false memory research by asking memory researchers what they considered to be the SESOI in false memory research. They were presented with three hypothetical and three influential paper scenarios. These scenarios depicted studies examining the effects of certain manipulations (e.g., therapy) on false memory formation using well-known false memory paradigms: Deese/Roediger-McDermott, misinformation, and forced fabrication. Subsequently, they were asked for each scenario what they would consider to be the SESOI for practical and theoretical purposes and justify their decisions. We found that there was no clear consensus for the SESOI. However, memory researchers tended to accept smaller SESOIs or "any difference that leads to a p < .05," especially for theoretical ends. We argue that the lack of a general consensus is acceptable as long as proper justification is used. We discuss such rationales and provide recommendations for setting the SESOI.

The importance of the smallest effect size of interest in expert witness testimony on alcohol and memory

“…Our position is that memory experts first need to decide and justify on what the smallest effect size of interest is in a given domain before the practical relevance of effect sizes can be discussed. Therefore, and in line with recent research ( Riesthuis et al, 2022 ), we argued that in the realm of applied memory research, a decrease or increase of 1 detail could be seen as the smallest effect size of interest.…”

“…To establish the smallest effect size of interest one can also question psychologists about what they regard as the smallest effect size of interest in their domain. In the first study of this kind in the field of memory, Riesthuis et al (2022) surveyed several memory researchers ( N = 41) to obtain the smallest effect size of interest for false memory research. Participants read several scenarios containing the method and procedure of frequently-used false memory paradigms, such as an experiment on the effect of therapy on the formation of false memories.…”

“…As sound as these ideas might be for the advancement of theory and identification of mechanisms, we suggest it does not work when it comes to rendering practical assistance in fields, such as in court. Taken this issue and the findings by Riesthuis et al (2022) into account, our proposal for the smallest effect size of interest in the area of applied memory research is that any factor—such as alcohol—that causes a gain or loss of even one correct or incorrect reported detail can be of significant practical importance (see Otgaar et al, 2022 ). One forgotten detail, or one incorrectly reported detail, could result in severe consequences in the courtroom.…”

Law and psychology must think critically about effect sizes

“…Determining this—the smallest effect size of interest—is a challenge. In one area of legal psychology, false memory research, respondents to a survey could not agree on the smallest effect size of interest and many conflated it with statistical significance [ 15 ]. But the task is easier and worth undertaking in publicly commissioned research with a clear goal of informing a specific change in law (versus studying false memories generally).…”

“…In this comment, I will examine a key methodological threat to the development of a public law and psychology—a failure to think critically about effect sizes. That is, it appears to not yet be common practice in law and psychology for researchers to (1) provide empirical or at least verifiable support for why effect sizes might matter, (2) to scrutinize effect sizes that are incredibly large, and (3) to plan studies [ 15 ] such that they are informative of whether a meaningful effect exists (i.e., practical relevance, see [ 13 , 14 ]). I analyze these issues with respect to influential studies in law and psychology, and then provide discipline specific guidance for moving forward.…”